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Second  Annual  Report 

Olfactory  Stimuli  for  Medical  Training  in  Virtual  Reality 
Myron  W  Krueger 
Artificial  Reality  Corporation 


Executive  Summary 

The  second  year  of  the  program  focussed  on  the  following  activities: 

1.  system  integration 

2.  odor  delivery  technology 

3.  odorant  acquisition 

4.  tracking  technology 

5.  odor  studies 

System  integration 

Odor  display  capability  was  integrated  into  an  immersive  virtual  reality 
system  with  a  head-mounted  display.  This  system  featured  wireless 
display  of  the  virtual  environment  as  well  as  wireless  control  of  the  odor 
presentation.  Demonstration  experiences  permitting  the  user  to  move  his 
head  around  to  sniff  the  virtual  objects  were  developed.  A  more  elaborate 
scenario  for  an  urban  accident  scenario  has  also  been  modelled. 

Odor  Delivery  Systems 

Two  preliminary  designs  of  advanced  delivery  systems  have  been 
developed.  One  of  these  was  specified  early  in  the  year  and  could  have 
been  constructed  immediately.  However,  given  what  we  have  learned,  we 
were  prudent  to  study  the  behavior  of  the  actual  odorants  in  more  detail 
before  committing  to  a  design.  A  second  family  of  advanced  designs  is  now 
being  prototyped  and  a  very  advanced  approach  not  contemplated  in  the 
original  proposal  has  been  considered. 

Odorant  Acquisition 

The  odors  required  for  medical  simulations  have  been  surveyed  and  proved 
to  be  much  more  poorly  understood  than  expected,  and  certainly  not 
available  commercially  as  our  original  odor  experts  at  Monell  Chemical 
Senses  Center  in  Philadelphia  had  believed.  We  have  sought  assistance 
from  the  companies  in  the  fragrance  industry,  a  free  lance  perfumer,  a 
chemistry  consultant,  and  environmental  air  quality  monitoring  firms. 

We  were  surprised  at  how  disappointing  some  of  the  industry’s  best 
efforts  were.  As  noted  earlier,  the  experience  of  the  industry  and 
perfumers  is  in  developing  aesthetically  pleasing  odors  while  many  of  the 
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odors  needed  by  this  project  are  either  unpleasant  or  neutral.  Of  the  few 
examples  of  such  odors  that  we  were  able  to  find,  only  those  that  have 
been  developed  over  many  years  are  really  convincing. 

From  the  air  quality  firms,  we  have  acquired  formulations  that  have  been 
used  for  testing  some  of  the  emergency  scenario  odors  that  we  will  need 
in  battlefield  casualty  scenarios,  e.g.  formulations  for  gasoline  and  diesel 
fuel  and  exhaust.  Burning  fuels,  explosives,  and  electrical  fires  are  still 
needed.  We  are  procuring  basic  odorant  materials  try  mixing  these  for 
ourselves. 

Wireless  Tracking 

Wireless  operation  is  required  to  allow  the  participant  to  experience 
odors  as  part  of  the  normal  exploration  of  a  virtual  environment.  While  the 
tracking  is  still  tethered,  progress  has  been  made  on  the  development  of  a 
wireless  technology.  Seven  alternative  approaches  have  been  analyzed.  One 
was  implemented  in  the  first  year,  and  two  others  are  under  development. 
Both  are  fast  and  accurate.  One  hinges  on  some  difficult  optics  which  are 
being  designed  by  Dr.  Jannick  Rolland  of  CREOL  (The  Center  for  Research 
and  Education  in  Optics  Laboratory  at  the  University  of  Central  Florida). 
The  initial  optical  designs  were  far  too  expensive  to  fabricate  but  several 
novel  compromises  have  been  considered  and  one  has  been  fabricated. 

Odor  Psychophysical  Studies 

We  focussed  on  technology  during  the  second  year  and  postponed  the 
studies  that  will  examine  the  value  of  odors  in  simulations  and  other 
human  interface-  applications.  To  this  end,  we  have  had  ongoing 
discussions  with  Joel  Warm  and  William  Dembers  at  the  University  of 
Cincinnati  who  have  performed  odor  studies  relating  to  military 
applications  in  the  past.  The  Pi’s  immediate  interest  is  in  assessing  the 
value  of  odors  as  alarms.  A  related  question  of  the  use  of  odors  to  provide 
cues  as  to  the  state  of  useful  but  not  critical  information  during  surgery. 

Third  Year 

In  the  final  year,  we  will  complete  the  wireless  tracking  system  with  an 
advanced  portable  odor  display  system.  We  will  continue  to  do  our  best  to 
acquire  and  develop  the  chemicals  needed  to  support  interesting  training 
scenarios,  although  this  is  a  difficult  process.  We  will  simulate  one 
environment  that  chosen  because  the  odors  needed  are  easy  to  obtain.  We 
will  also  do  a  battlefield  mass  casualty  scenario  and  one  for  elective 
surgery. 
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Report 


I.  Project  Summary 

The  activity  in  second  year  of  the  project  emphasized  the  technology 
development  and  postponed  some  of  the  behavioral  issues  while  we  sought 
to  increase  our  repertoire  of  odorant  materials  to  the  point  where  we 
could  support  meaningful  training  scenarios. 

The  main  areas  of  activity  were  as  follows; 

1.  System  integration 

2.  Delivery  system  development 

3.  Odorant  acquisition 

4.  Wireless  tracking  technology 

5.  Odor  studies 

II.  System  Integration 

The  most  significant  step  has  been  the  integration  of  the  various 
components  of  the  virtual  reality  system  to  create  a  system  in  which  odor 
display  is  incorporated  with  head  movement. 

The  participant  can  stand  in  a  simple  virtual  environment  and  experience 
the  odors  that  are  associated  with  a  number  of  objects  that  are  depicted 
in  that  environment.  As  the  person  moves  about  the  scene,  they  smell  the 
odor  of  each  object  that  they  approach.  The  strength  of  the  odor  depends 
on  the  distance  of  their  head  from  the  odor  stimulus. 

We  have  found  that  coupling  the  strength  of  the  odor  to  the  head  movement 
makes  the  experience  of  smelling  it  much  more  compelling.  In  situations 
where  an  odor  simply  permeates  a  scene,  we  find  that  odors  are  not  as 
convincing.  We  also  found  that  when  an  unpleasant  odor  is  displayed,  if  the 
person  cannot  reduce  the  sensation  by  moving  their  head  away  from  it, 
they  feel  that  they  are  trapped  in  the  head  mount  and  want  to  take  it  off. 
One  possible  explanation  for  this  is  that  odors  in  the  real  world  may  not 
be  homogeneously  distributed  around  the  environment  as  odor  research 
usually  assumes,  but  there  is  some  texture  to  the  odor  experience  making 
it  seem  real  and  an  odorized  mask  seem  less  so. 

Making  the  odor  display  respond  to  head  movement  requires  more  abrupt 
switching  from  one  odor  to  another  and  rapid  variation  of  the 
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concentration  of  the  selected  odor  as  the  head  is  moved.  This  feat  is 
accomplished  by  using  high  speed  valves  in  the  current  prototype. 
However,  it  also  depends  on  the  fact  that  current  delivery  schemes  have  a 
dedicated  evaporation  chamber  for  each  odorant,  so  we  are  not  flash 
evaporating  the  odorant  on  demand.  Finally,  concentration  is  being 
controlled  by  varying  the  duty  cycle  of  solenoid  valves  rather  than  by 
using  proportional  valves  to  vary  the  concentration  of  the  odorant  or  the 
air  flow. 

II. A.  Wireless  Operation 

Since  one  goal  is  to  allow  participants  to  walk  naturally  around  the 
virtual  world  and  encounter  odors  as  a  natural  consequence  of  their 
movement,  the  odor  delivery  system  was  made  wireless.  While  more  rapid 
switching  between  odors  is  ultimately  required,  we  initially  used  an  off- 
the-shelf  wireless  transmission  module  for  this  communication. 

Again,  since  a  completely  wireless  system  is  the  ultimate  goal,  it  was 
also  desirable  to  transmit  the  video  image  by  wireless  means  as  well. 
This  function  was  a  little  more  difficult,  because  while  the  first 
demonstration  system  we  used  worked  fine  with  a  stationary  receiver  in  a 
benign  laboratory  setting,  it  failed  miserably  when  the  participant  walked 
around  in  a  trade  show  environment  with  lots  of  other  electronic 
equipment  around. 

The  issue  that  we  had  naively  stumbled  on  was  well  known  to  television 
engineers  and  even  to  the  consumer  of  broadcast  as  opposed  to  cable- 
transmitted  television.  In  the  early  days  of  television,  it  was  common  for 
ghost  images  to  appear  on  the  screen.  These  were  the  result  of  the  signal 
reflecting  off  of  buildings  and  other  objects  in  the  neighborhood.  Since  the 
reflected  signal  travelled  a  greater  distance  than  the  main  signal,  it  was 
delayed  and  appeared  as  an  electronic  echo  on  the  screen-the  so-called 
“multipath”  problem.  By  adjusting  direction  that  the  antenna  was  facing, 
it  was  often  possible  to  aim  it  at  the  original  signal  and  ignore  the 
reflections. 

We  moved  from  low  cost  consumer  transmitter/receiver  pairs  up  to  the 
systems  used  by  network  cameramen.  These  systems  operate  at  a  2  Ghz 
frequency  as  opposed  to  the  900  Mhz  of  the  units  that  we  started  with, 
thereby  reducing  the  sensitivity  to  other  equipment.  However,  while  at 
first  glance,  it  would  seem  that  our  task  is  the  same  as  that  faced  by  a 
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cameraman — to  transmit  a  video  image,  there  are  two  major  differences. 
First,  wireless  cameras  are  often  used  outdoors  and  are  known  to  have 
problems  indoors.  Second,  the  camera  which  is  the  transmitter  is  mobile 
and  the  receiver  is  stationary—usually  in  a  nearby  van.  Virtual  reality 
presents  the  reverse  situation.  The  transmitter  is  stationary  and  the 
receiver— worn  by  the  participant— is  not  only  mobile,  but  it  is  also 
rotating  with  respect  to  the  transmitter.  Thus,  we  had  severe  multipath 
problems  even  with  the  high  frequency  system. 

The  solution  is  called  a  diversity  receiver.  Instead  of  a  single  receiver 
listening  to  the  signal,  this  technique  uses  multiple  antennas  and 
continually  chooses  the  one  receiving  the  strongest  signal.  As  the 
participant  turns,  the  signal  strength  entering  each  antenna  varies.  The 
first  such  system  we  used  was  much  better  than  the  single  antenna 
systems,  but  still  had  pronounced  glitches  as  the  person  turned  around. 
However,  by  moving  to  a  much  more  powerful  system,  we  were  able  to 
reduce  the  glitches  to  an  acceptable  faint  one  once  each  complete 
revolution.  Why  we  should  benefit  from  a  transmitter  which  can  send  for 
miles  when  we  are  only  going  a  few  yards  is  a  puzzle,  but  one  which  we 
became  incurious  about  as  soon  as  the  performance  was  deemed  adequate. 

One  consequence  of  our  current  solution  is  that  we  are  doomed  to  video 
resolution  for  the  moment  and  perhaps  for  the  foreseeable  future.  We  are 
not  aware  of  any  portable  commercial  product  that  will  transmit  VGA  or 
greater  resolution  with  or  without  diversity  receivers.  We  have  explored 
the  possibility  of  designing  such  a  capability  with  the  manufacturer  of  the 
current  system,  but  could  not  argue  that  there  is  an  immediate 
commercial  opportunity  for  such  a  system.  Indeed,  it  is  not  obvious  what 
application  other  than  virtual  reality  would  feel  the  need  and  unlikely  that 
the  virtual  reality  community — or  what  is  left  of  it — would  embark  on  the 
development  of  such  a  capability  until  the  video  resolution  device  had 
proven  itself  in  the  marketplace. 

II. B.  Odor  Scenario 

To  test  the  odors  that  we  are  acquiring  and  the  delivery  capability  we  are 
developing,  we  have  modelled  an  urban  auto  accident  scenario  into  which 
we  can  insert  our  odors  as  they  become  available.  At  a  downtown 
intersection,  one  car  runs  a  traffic  light  and  collides  with  another  vehicle 
which  is  entering  the  intersection  legally  and  then  strikes  a  pedestrian 
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who  is  crossing  the  street. 


The  scene  chosen  was  civilian  rather  than  military  primarily  because  we 
had  more  of  the  odors  we  would  need  to  depict  this  scene.  (The  odors  on 
hand  vary  greatly  in  their  effectiveness.)  Also,  the  graphics  modelling 
required  to  illustrate  a  street  intersection  is  easier  to  because  the  scene 
consists  mainly  of  rectangular  surfaces  which  are  fairly  close  to  the 
viewer  and  can  be  rendered  quickly. 

The  scene  that  results  from  the  accident  contains  a  considerable  number 
of  opportunities  for  odors  to  be  displayed.  The  offending  car  bursts  into 
flame  and  the  guilty  driver  staggers  out  of  it  and  falls  on  the  ground. 
Since  his  leg  has  been  penetrated,  he  is  suffering  from  arterial  bleeding. 
Blood  spurts  from  his  wound  and  pools  alongside  his  body.  The  large 
surface  area  of  the  blood  makes  the  smell  perceptible  although  near  the 
individual’s  face,  its  rather  subtle  odor  is  eclipsed  by  the  alcohol  on  his 
breath.  Nearby  trash  cans  that  were  knocked  over  by  the  accident  add  to 
the  potpourri  of  odors. 

The  pedestrian’s  abdomen  has  been  penetrated  by  some  part  of  the  car  and 
smells  of  stool  from  a  punctured  bowel  as  well  as  from  blood.  The  driver 
of  the  second  car  has  suffered  from  a  bloody  but  superficial  head  trauma. 
However,  since  he  is  on  his  way  back  from  jogging  in  the  park,  he  smells 
of  sweat.  Behind  his  car,  fuel  from  a  ruptured  gas  tank  is  puddling  on  the 
ground.  As  time  passes  after  the  accident,  it  seems  unlikely  to  ignite. 

Shortly  after  the  accident,  a  squad  car  and  an  ambulance  arrive  and  their 
engines  are  left  running.  Anyone  moving  near  the  rear  of  these  vehicles 
will  smell  their  exhausts.  The  squad  car,  however,  does  not  smell  of 
donuts — yet. 

The  participant  is  cast  in  the  role  of  a  single  paramedic  working  alone  (We 
acknowledge  that  they  usually  work  in  pairs.)  faced  with  a  classic  triage 
situation  because  of  the  multiple  casualties.  He  approaches  the  scene 
from  the  direction  of  the  second  car.  A  quick  conversation  with  its  driver 
confirms  that  this  individual  is  not  hurt  seriously  and  can  be  skipped  for 
the  moment. 

He  then  approaches  the  injured  pedestrian  who  is  lying  on  the  ground. 
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Along  the  way,  he  is  subjected  to  the  smell  of  pizza  wafting  from  a  nearby 
restaurant.  As  he  bends  over  this  individual,  he  smells  the  fecal  odor 
coming  from  the  wound,  but  a  quick  check  of  this  person’s  vitals  indicates 
that  his  wounds  while  serious  will  not  kill  him  in  the  next  few  minutes. 

As  the  participant  reaches  the  drunken  driver,  he  notes  the  odor  of  the 
trash  spilled  around  him  and  the  alcohol  on  the  victim’s  breath.  He  then 
attends  to  the  serious  bleeding  from  the  leg.  Once  this  bleeding  is  stopped, 
this  victim  has  been  stabilized  and  the  paramedic  returns  to  the 
abdominal  injury. 

During  this  scenario,  it  is  assumed  that  the  paramedic  trainee  would  be  in 
verbal  communication  with  a  remote  trainer  or  with  a  remote  physician 
who  can  monitor  his  actions  through  a  simulated  head-mounted  video 
camera.  Such  teleconsulting  could  improve  the  level  of  care  administered 
at  the  accident  site  and  could  allow  ER  personnel  to  have  a  better  idea  of 
what  to  expect. 

II. C.  Odor  Behavior 

In  the  development  of  this  scenario,  it  became  evident  that  each  odor 
might  behave  differently,  both  because  of  differing  physical  properties 
and  because  of  one’s  sensitivity  to  it.  For  instance,  blood,  gasoline,  diesel 
fuel,  etc  will  only  be  smelled  if  the  person  is  close  to  the  source  or  if  the 
surface  area  is  large.  Sweat,  alcohol,  and  trash  smells  will  also  probably 
only  be  perceptible  near  their  sources.  However,  the  smell  of  the  fire  will 
be  affected  by  the  wind  and  travel  through  the  scene  in  gusts.  The  pizza 
odor,  on  the  other  hand,  comes  from  a  very  large  source  and  may  be 
perceptible  through  a  large  part  of  the  scene,  although  it  could  also  be 
affected  by  the  wind. 

It  is  necessary  to  consider  these  influences  when  displaying  odors. 
However,  the  behavior  of  the  different  odors  in  the  real  world  in  different 
atmospheric  conditions  has  not  been  observed  upon  let  alone  recorded. 
Until  instrumentation  as  good  as  the  human  nose  can  analyze  real  world 
environments  to  tell  us  what  we  have  available  to  smell,  we  will  have  to 
rely  on  artistry  based  on  personal  observation  and  experience.  However, 
while  we  do  not  know  exactly  how  any  given  odor  behaves,  we  can  identify 
the  types  of  behavior  that  might  be  possible  and  build  the  ability  to 
simulate  these  behaviors  into  our  simulations  even  if  we  do  know  exactly 
how  to  assign  the  parameters  to  any  given  odor. 
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In  general,  we  can  predict  the  need  to  model  atmospheric  conditions  and 
circulation  in  virtual  reality.  We  need  to  know  how  quickly  each  odorant— 
and  each  of  its  components — will  evaporate  under  any  set  of  conditions  of 
temperature,  barometric  pressure,  humidity,  etc.  We  need  to  know  how 
long  each  substance  will  remain  airborne  and  in  what  concentrations 
people  can  detect  it. 

We  will  then  need  to  model  the  circulation  of  the  air,  the  influence  of  the 
local  terrain  and  buildings  on  the  wind,  heat  from  the  sun  and  fires,  and 
even  precipitation  if  we  are  to  complete  the  virtual  reality  experience. 
While  detailed  air  circulation  models  are  beyond  the  scope  of  this  project, 
the  Pi’s  experience  of  doing  5  years  of  gas  flow  visualization  under 
contract  with  Pratt  and  Whitney  give  him  a  clear  idea  of  what  is  involved 
in  such  simulation.  He  can  even  imagine  robotic  mobile  fans  and  heaters 
surrounding  the  participant  that  would  move  in  concert  with  him  and 
provide  the  appropriate  sensations  moment  by  moment. 

For  the  moment,  we  are  content  to  do  what  the  graphics  community  has 
done  for  depicting  smoke  and  flames.  They  have  simulated  the  appearance 
but  not  the  detailed  physics  of  these  phenomena  (which  are  not  completely 
understood).  In  the  inner  city  accident  scenario,  we  had  a  graphic  mode  for 
explicitly  showing  the  odor  fields.  The  odors  from  the  humans  were  shown 
as  static  plumes  rising  from  the  sources.  These  fields  did  not  reach  far 
from  the  sources  because  such  odors  are  not  strong  and  would  dissipate 
quickly.  In  addition,  there  was  enough  of  a  breeze  depicted  to  dilute  these 
odors  down  below  detectable  levels. 

III.  Delivery  System  Development 

In  the  first  year,  odor  delivery  systems  for  three  different  virtual  reality 
formats  were  developed:  the  head-mounted  display,  a  surgical  training 
booth,  and  a  CAVE.  As  a  result  of  this  experience,  the  CAVE  format  was 
abandoned  because  the  CAVE  space  is  very  difficult  and  very  expensive  to 
odorize  uniformly  and  to  change  rapidly.  In  addition,  no  means  of 
selectively  odorizing  the  space  within  the  CAVE  itself  was  conceived. 
This  meant  that  while  the  environment  surrounding  the  CAVE  could  have  an 
odor,  perversely  objects  which  seemed  to  be  in  the  CAVE  could  not  have 
odors  localized  to  them.  The  decision  was  made  to  recommend  the  same 
delivery  system  used  with  the  HMD  to  be  worn  in  the  CAVE. 
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Currently,  it  is  seen  as  desirable  if  the  delivery  system  being  developed 
for  the  HMD  can  also  be  for  the  surgical  training  booth.  It  is  expected  that 
the  preliminary  versions  of  that  device  which  are  not  yet  miniaturized  can 
be  used  in  that  format.  Only  the  volume  of  air  odorized  and  the  amount  of 
odorant  used  will  differ.  However,  the  dilution  of  the  odorized  air  by  the 
ambient  air  in  the  booth  reduces  its  strength  by  a  factor  of  ten  compared 
to  breathing  it  through  a  mask. 

One  change  has  been  made  in  the  architecture  of  the  surgical  training 
booth.  Originally,  it  was  thought  that  we  could  introduce  the  odors  by 
blowing  them  gently  in  the  user’s  face  and  that  the  positive  pressure 
would  guarantee  that  he  smelled  only  the  newly  introduced  air  and  not  any 
odors  that  had  been  introduced  earlier.  A  ceiling  exhaust  system  then 
evacuated  the  odors  on  a  continual  basis.  While  this  approach  works,  we 
were  concerned  that  the  forced  air  in  the  face  might  bother  some  people 
and  that  residual  air  which  collected  at  the  bottom  of  the  booth  would 
never  be  evacuated  causing  the  booth  to  absorb  the  odor.  The  only  way  to 
assure  its  elimination  would  be  to  exhaust  air  at  the  bottom  of  the  booth 
as  well  as  the  top.  While  the  above  system  had  the  benefit  of  requiring 
minimal  changes  to  a  standard  library  carrel,  we  decided  to  consider  a 
different  approach. 

The  new  technique  is  to  release  the  odors  in  front  of  the  user’s  face  as 
before,  but  not  under  pressure.  Instead,  a  panel  of  exhaust  holes  is 
mounted  behind  his  head.  A  suction  greater  than  the  capacity  of  the  holes 
assures  a  uniform  evacuation  of  odors  across  its  surface.  The  result  is  a 
laminar  flow  from  in  front  of  the  user’s  face  to  behind  his  head.  Since  the 
exhaust  panel  is  not  very  thick,  it  can  be  mounted  in  an  existing  carrel  and 
vented  out  the  ceiling  as  before. 

III.A.  Advanced  Delivery  System  Options 

In  the  first  year,  a  variety  of  odor  delivery  schemes  were  considered.  In 
the  second  year,  a  number  of  these  were  evaluated  and  a  main  path  chosen. 
Prototyping  along  that  path  has  progressed  and  a  twelve  odor  wireless 
system  has  been  fabricated  and  is  being  tested. 

III.A. 1.  General  Delivery  Architecture 

An  odor  delivery  system  can  consist  of  the  following  elements: 

1.  an  odor  storage  technique 
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2.  a  means  of  converting  the  stored  odor  to  a  vapor 

3.  a  means  of  introducing  that  vapor  into  a  breathable  air  stream 

4.  a  means  of  delivering  the  odorized  air  to  the  nose 

III.A.2.  Odor  Storage 

Several  odor  storage  techniques  have  been  examined  including  the 
following: 

1.  liquids 

2.  waxes 

3.  impregnated  polymers 

4.  microencapsulation 

In  examining  the  virtues  and  disadvantages  of  each  of  these  storage 
approaches,  we  discovered  that  the  storage  technique  was  not  so 
important  in  and  of  itself  as  what  it  implied  about  the  operation  of  the 
rest  of  the  system  and  the  cost  of  preparation. 

For  instance,  odorants  are  most  often  distributed  as  liquids.  The  odorants 
that  are  incorporated  into  other  storage  media  are  sold  in  liquid  form. 
Therefore,  basing  a  delivery  system  on  liquids  would  eliminate  a 
processing  step. 

In  addition,  odorants  which  are  encapsulated  or  impregnated  in  other 
materials  must  be  released  from  those  carrier  substances  before  they  can 
be  vaporized.  When  they  are  released,  there  are  two  problems.  One  is 
assuring  that  all  of  the  odorant  ingredients  are  released  simultaneously. 
The  second  is  the  residue  of  the  carrier  material  that  remains  after  the 
odorant  has  been  released. 

In  the  case  of  microencapsulation,  each  microcapsul  has  to  be  broken  if 
its  odor  is  to  be  released.  This  rupturing  can  be  accomplished  by  heat  or  by 
mechanical  pressure.  In  either  case,  the  microcapsul  cannot  release  its 
odor  instantaneously.  It  will  continue  to  smell  until  all  of  its  odor  has 
evaporated.  Also,  while  microcapsuls  are  quite  stable,  they  do  leak,  i.e. 
given  a  large  number  of  unbroken  capsuls,  one  can  smell  the  odorant  they 
contain.  The  release  of  odor  before  and  after  it  is  actually  desired  implies 
that  microcapsuls  would  have  to  be  broken  and  their  odor  released  in  a 
chamber  apart  from  the  air  flow  and  valves  would  select  which 
evaporation  chamber  would  be  selected. 
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The  alternative  is  to  live  with  the  leakage  or  to  permit  the  user  to 
actually  inhale  the  ruptured  microcapsuls  which  are  extremely  small  and 
made  of  benign  materials.  While  this  practice  is  probably  safe  given  the 
short  exposures,  it  seems  to  be  cavalier  to  ask  users  to  take  anything  into 
their  bodies  that  they  do  not  absolutely  have  to.  Another  problem  with 
microencapsulation,  of  course,  is  the  expense  (about  $5K  per  odorant)  of 
the  process  which  has  to  be  tailored  to  each  material,  in  addition,  the 
leakage  implies  that  these  materials  cannot  be  stored  indefinitely. 

III.B.  Odor  Delivery  Designs 
III.B.1.  Micro-Hotplates 

In  the  case  that  each  odorant  has  its  own  evaporation  chamber,  it  is  likely 
that  the  different  components  of  each  smell  will  be  released 
differentially.  In  an  approach  which  Lawrence  Livermore  Laboratories 
assured  us  they  could  realize,  each  odorant  would  be  stored  as  a  single 
droplet  deposited  on  a  micro-hotplate.  In  the  case  where  the  storage  and 
evaporation  chambers  are  identical,  valves  are  required  to  release  the 
vapor  from  each  of  these  chambers.  One  problem  with  these  micro¬ 
hotplates  is  that  it  is  not  clear  how  well  they  will  work  for  the  complex 
mixtures  of  materials  which  are  typical  of  real  world  odors.  In  this  case, 
the  most  volatile  materials  will  evaporate  first,  cooling  the  heater.  Over 
time,  the  odor  released  from  the  droplet  will  change  character  until  only 
the  least  volatile  component  remains. 

The  way  to  counteract  this  is  to  store  the  odorant  in  a  second  chamber  and 
to  deposit  a  much  smaller  droplet  on  the  hotplate  which  can  be  evaporated 
completely  before  the  next  droplet  is  deposited.  One  problem  with  this 
approach  is  that  it  is  more  complex,  because  individual  valves  must  be 
provided  for  each  hotplate  as  well  as  a  means  for  depositing  the  droplets 
on  it.  In  addition,  there  is  a  dilemma  in  getting  the  vapor  from  the 
hotplate  to  the  main  air  stream.  If  the  hotplate  is  in  a  separate  chamber, 
valves  are  again  required  to  release  the  vapor.  If  the  hotplate  is  in  or  near 
the  main  air  stream,  there  is  the  danger  that  that  air  will  cool  the 
hotplate.  The  hotplates  would  work  if  they  could  perform  the  valving 
function  as  well,  releasing  the  vapor  they  create  into  the  main  air  stream. 
However,  it  is  likely  that  residue  from  the  evaporation  process  would  foul 
the  hot  plates  over  time. 

III.B. 2.  Piezo-Electric  Vaporization 
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Another  approach  that  we  investigated  was  the  use  of  piezo-electric 
crystals  to  accelerate  evaporation.  Dr  Thomas  Avedisian  of  the  Mechanical 
Engineering  Department  at  Cornell  performed  a  study  of  this  approach  for 
us  on  a  consulting  basis.  His  report  is  included  as  Appendix  A.  He  was  able 
to  use  the  vibration  of  a  piezo  crystal  to  rupture  a  droplet  so  that  it  would 
create  much  smaller  droplets.  While  this  approach  worked,  it  could  not  be 
very  finely  tuned.  It  also  required  a  means  of  depositing  droplets  on  the 
crystal  and  then  a  means  of  getting  the  droplets  into  the  airstream.  Once 
in  the  airstream,  it  would  be  necessary  to  guarantee  that  they  would  be 
completely  evaporated  before  they  reached  the  nose. 

III.B.3.  Piezo-Electric  Films 

Another  small  study  was  performed  using  piezo-electric  films  instead  of 
crystals  by  Dr.  Martin  Fox  of  the  Electrical  Engineering  Department  of  the 
University  of  Connecticut  who  has  long  experience  with  fabricating 
ultrasonic  transducers  for  biomedical  applications.  In  this  case,  higher 
frequencies  could  be  used  and  vapors  as  opposed  to  droplets  were  created. 
As  with  the  crystals,  there  was  a  problem  with  how  to  deposit  droplets 
onto  the  film.  Initially,  the  hope  was  that  we  could  wick  material  onto  the 
film  from  a  reservoir.  Piezo  films  could  then  be  used  to  fabricate  pumps, 
fans,  and  valves  out  of  the  same  technology.  The  advantage  would  be  that 
such  a  system  would  potentially  be  very  inexpensive — even  disposable. 
While  such  an  approach  could  probably  have  been  made  to  work,  it  would 

have  required  much  more  support  than  we  were  able  to  get  from  the 

manufacturers  of  these  materials.  As  usual,  there  would  be  a  significant 
tooling  cost  in  order  to  achieve  low  cost  production. 

III.B.4.  Ultrasonic  Nozzle 

Another  approach  we  tried  was  to  repurpose  an  ultrasonic  nozzle  typically 
used  in  spray  painting  manufactured  by  Sono-Tek  Corporation  of  Milton, 
New  York.  This  is  similar  to  the  vibrating  piezo  films  except  that  it  is 
done  at  a  much  higher  frequency.  In  this  case,  cylindrical  piezoelectric 
actuators  vibrate  a  titanium  tube  at  120  Khz  causing  a  very  fine  mist  of 
tiny  droplets  to  be  released  from  its  end.  While  this  approach  might  have 
been  ideal  for  odorizing  a  room,  the  quantity  of  material  released  was  too 

large  to  be  of  interest  for  either  a  surgical  simulation  booth  or  a  wearable 

odor  display.  It  is  quite  possible  that  the  approach  could  have  been  scaled 
down,  but  the  product  we  tested  was  addressing  a  specific  market  and  the 
manufacturer  did  not  have  the  skills  required  to  miniaturize  it.  Also, 
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while  the  mist  produced  was  exceedingly  fine,  it  still  comprised 
individual  droplets  which  we  would  be  reluctant  to  feed  directly  into 
someone’s  nose  without  converting  it  into  a  vapor  first.  Thus,  even  with 
miniaturization,  this  approach  would  require  another  stage  after  it.  Also, 
we  were  not  able  to  turn  it  on  and  off  instantly. 

III.B.5.  Air  Stream  Vaporization 

The  last  idea  is  an  example  of  a  design  approach  that  has  informed  our 
recent  development.  This  approach  is  based  on  the  following  conclusions. 

1.  it  is  preferable  to  store  the  odors  as  liquids 

2.  it  is  advantageous  to  somehow  convert  the  liquid  odorant  into  a 
vapor  at  the  point  where  it  is  introduced  into  the  main  air  stream. 

3.  It  would  also  attractive  if  the  liquid  could  be  released  into  the  air 
stream  and  vaporized  there  or  released  as  a  vapor  rather  than  as  droplets. 

III.B.6.  Micro-Droplet  Formation 

Microencapsulated  materials  have  a  discrete  quality  in  that  it  is  not 
possible  to  crush  half  a  capsule.  Similarly,  droplet  based  systems  have  the 
droplet  itself  as  their  quantum.  We  have  focussed  on  individual  droplet 
formation,  because  it  is  quantifiable.  If  we  know  the  concentration  of  the 
liquid,  the  size  of  the  droplet,  and  the  volume  of  air  that  it  is  vaporized 
into,  we  know  the  concentration  of  the  odorant  in  the  air  that  the 
participant  is  breathing. 

While  it  might  be  possible  to  vary  the  dilution  of  the  liquid  as  it  is 
dispensed  or  to  vary  the  volume  of  the  air  that  it  is  mixed  with,  doing 
such  precision  mixing  on  the  fly  seems  to  be  an  undesirable  complexity. 
Therefore,  we  are  working  with  a  fixed  dilution  and  a  fixed  droplet  size. 
We  can  vary  the  concentration  of  the  odorant  in  the  air  by  controlling  the 
number  of  droplets  released  per  unit  time. 

We  have  tried  a  number  of  methods  for  creating  single  droplets.  One  is  to 
feed  odorants  through  a  very  small  glass,  stainless  steel,  or  teflon 
pipette.  The  pipette  is  attached  to  a  piezo-electric  crystal.  When  the 
crystal  is  oscillated  at  its  resonant  frequency,  it  shakes  loose  a  steady 
stream  of  tiny  (100  nl)  droplets. 

Our  hope  was  that  shaking  loose  one  droplet  would  cause  its  volume  to  be 
replaced  in  the  pipette  by  capillary  action.  This  did  not  prove  to  be  the 
case.  A  small  positive  pressure  was  required  to  keep  the  pipette  filled.  We 
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also  hoped  that  we  could  give  the  crystal  a  single  pulse  and  shake  loose  a 
single  droplet.  Because  the  pipette/crystal/odorant  system  has  a  fixed 
resonant  frequency,  several  droplets  are  typically  released. 

However,  we  have  found  that  if  we  keep  the  pipette  oscillating,  no 
droplets  will  be  released  unless  we  force  them  out.  By  providing  very 
precise  control  of  the  flow  into  the  pipette,  we  gain  greater  control  of  the 
number  of  droplets  that  are  released.  Initially,  we  controlled  of  the  flow 
by  using  a  linear  actuator  to  advance  the  plunger  of  a  syringe  filled  with 
odorant.  Since  the  actuator  was  geared  way  down,  very  small  numbers  of 
droplets  could  be  dispensed. 

Two  methods  of  vaporizing  the  odorant  have  been  examined.  The  first  is  to 
simply  let  the  droplets  fall  onto  a  hotplate  which  will  cause  them  to 
evaporate.  Since  it  takes  a  little  time  for  the  odorant  to  evaporate  and 
there  is  the  possibility  of  a  residue  building  up,  we  thought  about  making 
the  hotplate  rotate  to  assure  that  droplets  were  always  falling  on  a  clean 
surface. 

III.B.7.  Micro-Venturi  Vaporization 

The  other  technique  is  to  put  the  droplets  into  a  micro-venturi  system. 
This  device  is  a  carefully  formed  tube  whose  inside  diameter  is  rapidly 
constricted  and  then  gradually  expanded.  The  result  is  that  turbulence  is 
maximized  and  therefore  the  droplet  mixes  uniformly  with  the  air  stream. 
Actually,  the  droplet  is  dispersed  rather  than  evaporated.  The  venturi 
requires  a  minimum  flow  rate  to  operate  (5  liters  per  minute)  and  a 
source  of  pressurized  air  (5-10  psi).  Originally,  we  have  had  the 
participant  wear  a  compressed  air  bottle  strapped  to  his  thigh.  Recently, 
we  have  started  to  use  a  pump  to  provide  the  pressure,  because  it  is 
lighter.  Initially,  the  noise  of  the  pump  was  communicated  through  the  air 
flow  to  the  participant’s  nose  and  through  the  eustachian  tube  to  his  ears 
where  it  was  perceived  as  sound.  Since  then,  quieter  pumps  have  been 
found  which  operate  from  batteries. 

The  system  as  described  only  delivers  a  single  odor.  However,  we  found 
that  we  could  drill  up  to  6  ports  into  the  output  of  the  venturi.  Each  of 
these  operates  as  described.  In  addition,  the  output  of  several  verturis  can 
be  fed  into  the  input  of  yet  another  venturi  for  further  mixing.  We  have 
currently  done  this  for  two  venturis  with  six  odors  each  for  a  total  of 
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twelve  odors. [Appendix  F] 


We  thought  it  might  be  possible  to  expand  the  number  of  odors  by  forming 
an  antichamber  outside  the  venturi  which  is  connected  to  its  output  by  a 
narrow  channel.  We  hoped  the  vacuum  from  the  venturi  would  suck  any 
droplets  introduced  to  the  antichamber  into  the  venturi.  A  little  bleed  air 
was  introduced  to  the  antichamber  to  cleanse  it.  We  thought  that  up  to  six 
channels  could  be  fed  into  each  antichamber  for  a  total  of  36  odors,  but 
that  approach  did  not  work  as  well  as  we  hoped. 

While  the  shaking  loose  of  droplets  is  very  easy  to  understand,  the  fact 
that  the  piezo  crystals  needed  room  to  oscillate  and  required  high 
voltages  to  operate  added  to  the  complexity  of  the  design,  so  we 
experimented  with  the  idea  of  pumping  liquid  directly  into  the  outlet  of 
the  venturi. 

If  we  could  pump  liquid  directly  into  the  venturi  without  forming  droplets, 
then  we  could  imagine  an  array  of  micropumps  or  valves  that  would  drive 
all  of  the  odorant  channels.  In  this  case,  the  odorant  channels  could  be 
molded  or  routed  out  of  the  same  piece  of  material  that  the  venturi  itself 
in  fabricated  from.  The  pump  array  could  then  be  clamped  onto  these 
channels  in  one  operation.  In  turn,  an  array  of  reservoirs  could  be  clamped 
onto  the  input  to  the  pump  array.  We  were  initially  very  enthused  by  a  10 
mm  X  10  mm  x  2  mm  self-priming  micropump  from  IMM,  a  german 
corporation  based  in  Mainz.  However,  its  output  turned  out  to  be  too  low 
for  our  purposes,  it  was  too  slow,  and  its  multiple  high  voltage  power 
supplies  presented  an  additional  design  challenge. 

Instead,  we  turned  to  high  speed  liquid  valves  from  Lee  Corporation.  We 
tried  the  idea  of  pumping  the  liquid  directly  into  the  outlet  of  the  venturi 
and  found  that  we  were  able  to  meter  small  amounts  of  liquid  by 
controlling  the  time  that  the  valves  were  open  in  1/2  millisecond 
increments.  When  the  valves  are  open,  odor  is  released.  When  the  valves 
are  closed,  no  odorant  appears  to  be  drawn  into  the  airflow  by  suction. 
(The  absence  of  unwanted  odorant  release  was  previously  noted  with  the 
micropipettes.)  By  simply  controlling  the  flow  of  liquid  to  the  venturi,  we 
are  no  longer  forming  droplets.  Exactly  in  what  form  the  liquid  odorant 
enters  venturi,  we  cannot  know.  It  may  bulge  out  from  the  outlet  of  the 
channel  into  the  venturi  and  be  torn  into  droplets  by  the  turbulence  or  it 
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may  rapidly  evaporate  in  the  suction  of  the  venturi. 
lll.C.  Wireless  Operation 

A  twelve  odor  odor  delivery  system  based  on  the  liquid  valves  has  been 
been  developed  which  can  operate  at  submillisecond  rates  when  tethered. 
Making  it  wireless  was  more  of  a  problem  than  it  had  been  for  the  slower 
demonstration  in  the  previous  year. 

While  the  system  worked  fine  on  the  bench,  each  step  in  incorporating 
wireless  control  with  portable  operation  ran  into  difficulties.  Enclosing 
the  unit  in  the  carrying  case  and  letting  the  receiver  rotate  as  the 
participant  walked  around  made  the  antenna  selection  much  more  critical. 
In  addition,  there  were  many  more  transmission  errors  than  before. 

To  combat  these  problems,  a  complex  error  checking  and  correction 
scheme  was  implemented  with  an  onboard  microprocessor  and  the  results 
of  each  transmission  were  sent  back  to  the  host  computer.  Unfortunately, 
this  procedure  was  itself  error  prone.  Experimentation  showed  that  the 
transmitter  and  receiver  could  not  be  operated  simultaneously  if  they 
were  in  close  proximity  as  they  are  in  the  portable  system.  It  was 
therefore  necessary  to  shut  off  one  of  these  functions  when  the  other  was 
operating.  It  was  further  discovered  that  these  devices  had  to  be  on  for  a 
considerable  time  before  they  could  transmit  or  receive  reliably.  Thus,  the 
wireless  function  only  worked  when  the  data  rates  were  slowed  down 
enough  that  we  were  no  longer  meeting  our  performance  goals. 

The  solution  was  to  abandon  the  elaborate  error  checking,  correction,  and 
reporting  capability  during  interactive  sessions.  Instead,  data  is  simply 
received  by  the  portable  unit  and  acted  upon.  If  the  perceived  behavior  is 
not  what  is  expected,  the  error  checking  is  turned  on  for  health  testing 
and  maintenance. 

The  valves  used  in  this  delivery  system  worked  fine  during  initial  testing. 
However,  over  time  we  noticed  that  some  of  the  valves  became  clogged. 
Lee,  the  manufacturer  of  the  valves,  took  them  apart  and  found  that  the 
coatings  had  swelled  from  extended  exposure  to  the  odorant  liquids.  To 
combat  this  problem,  we  chose  a  more  expensive  valve  with  a  more  inert 
coating  and  modified  the  delivery  system  so  that  we  could  flush  the 
valves  with  water  for  an  extended  period  after  each  use. 
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Lamentably,  the  problem  has  resurfaced.  After  months  of  use,  even  with 
flushing,  the  valve  coating  reacted  to  the  odorant  materials.  As  a 
consequence,  we  have  ordered  the  most  expensive  valves  with  a  Viton™ 
coating  in  the  hopes  that  this  material  will  withstand  the  odorants.  Lee  is 
providing  us  with  samples  of  their  coating  materials  for  us  to  expose  to  a 
variety  of  our  odorants.  They  will  then  analyze  these  samples  to  see  how 
they  have  reacted  to  the  odorants.  It  should  be  pointed  out  that  even  the 
most  seemingly  benign  odorant  materials,  e.g.  vanilla  and  peppermint, 
ultimately  damaged  the  valves.  If  a  single  coating  cannot  be  used,  it  would 
mean  that  odorants  would  have  to  be  matched  to  specific  valve  coating, 
making  the  system  much  less  flexible. 

III.D.  Microminiature  Design  Approaches 
III.D.1.  Ink  Jet  Print  Heads 

The  current  delivery  technique  releases  tiny  droplets  into  an  airstream 
where  they  are  vaporized  in  a  micro-venturi  system.  If  it  were  possible  to 
vaporize  the  liquid  odorant  before  it  was  released  into  the  airstream,  the 
delivery  system  could  consist  of  just  a  vapor  release  system  mounted  in 
front  of  the  nose.  The  air  pump,  the  valves,  the  micro-venturi  apparatus, 
and  the  tubing  from  the  back  to  the  nose  could  all  be  dispensed  with. 

Bubble  jet  printers  almost  do  what  is  desired  in  that  they  heat  ink  to  its 
boiling  point  creating  a  rapidly  expanding  bubble  that  forces  a  tiny  droplet 
of  ink  out  of  the  nozzle.  When  the  heater  is  turned  off,  the  bubble 
collapses  and  the  firing  chamber  refills  with  ink.  It  would  seem  that  this 
technique  could  be  modified  to  release  the  vaporized  liquid  instead  of  the 
droplets.  We  have  seen  this  possibility  since  the  beginning  of  the  program 
and  have  engaged  a  number  of  consultants  to  look  at  this  problem  for  us.  In 
each  case,  initial  enthusiasm  gives  way  to  the  frustrations  of  reverse 
engineering  the  printer,  because  of  the  lack  of  documentation  about  how 
they  operate  and  the  refusal  of  the  manufacturers  to  provide  any  technical 
information. 

The  investigations  of  our  consultants  and  discussions  with  a  firm  that 
makes  its  own  printhead  gave  us  enough  information  to  see  what  the 
problems  would  be  if  we  successfully  converted  an  ink  jet  printer  to 
deliver  odorant  liquids. 

III.D. 2.  Ink  Jet  Printers 
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The  simplest  ink  jet  printers  channel  a  single  liquid  ink  to  up  to  64 
nozzles  (1000  in  one  product).  Color  printers  have  basically  have  separate 
heads  for  each  color  as  well  as  black.  Each  nozzle  releases  a  25  micron 
droplet  every  25  microseconds.  This  technology  is  carefully  tuned  to  the 
physical  properties  of  the  ink  and  the  need  to  create  perfectly  formed 
droplets  in  the  output.  We  have  been  looking  at  how  we  might  repurpose 
this  technology  to  deliver  odors. 

Ink  jet  print  heads  consist  of  a  sandwich  of  three  layers: 

1.  the  electronics  layer  which  contains  the  heaters  for  the  bubble  jet 
printers  or  the  actuators  for  the  piezo-electric  printers. 

2.  the  polymer  via  layer  containing  the  channels  that  distributes  the  ink 
to  each  of  the  nozzles 

3.  the  nozzle  layer  from  which  the  ink  is  actually  ejected. 

To  this  end,  we  attempted  reverse  engineering  ink  jet  printers  to  see  if 
we  could: 

1.  get  it  to  deliver  odorants  instead  of  ink 

2.  figure  out  how  to  modify  the  via  layer  to  deliver  a  different  odorant 
to  each  nozzle. 

3.  figure  out  how  to  connect  the  via  channels  for  the  individual 
odorants  to  separate  reservoirs  for  each  odorant. 

The  appeal  of  the  ink  jet  print  head  is  that  it  is  the  only  electromechnical 
dispenser  in  mass  production.  At  least  one  hundred  million  have  been  sold. 
However,  the  essence  of  mass  production  is  that  enormous  engineering  and 
tooling  costs  have  been  incurred  in  making  the  technology  do  exactly  what 
it  does.  The  companies  involved  have  no  interest  in  ancillary  markets  that 
are  anything  but  huge.  For  instance.  Trident  International  of  Brookfield 
Connecticut  a  manufacturer  of  a  custom  print  head  for  marking  packages 
has  $200M  in  sales  and  plans  to  reach  $5B  in  just  a  few  years.  They  have 
no  interest  in  a  non-printing  application.  Even  the  companies  that  fill  or 
refill  cartridges  with  ink  could  not  be  persuaded  to  provide  empty 
cartridges  that  we  could  fill  with  odorants  instead. 

Ink  jet  printer  technology  appears  to  point  to  the  possibility  of  a 
miniature  dispensing  devices  for  liquid  odorants,  but  the  fact  that 
existing  heads  can  only  print  one  odorant  for  each  color  of  ink  means  that 
a  number  of  heads  would  be  required  to  support  a  meaningful  repertoire.  In 
addition,  these  heads  are  not  so  small,  are  not  inexpensive,  and  require 
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high  voltage  power  supplies. 


There  are  however  many  separate  nozzles  in  the  print  head  each  of  which 
in  principle  could  be  used  to  deliver  a  separate  odor.  This  is  a  problem 
that  none  of  the  printer  companies  have  ever  addressed.  For  them,  the 
trick  is  to  get  the  via  layer  in  the  print  head  to  channel  the  same  color  ink 
to  each  of  the  nozzles.  They  have  legions  of  PhD  chemists  who  control  the 
physical  properties  of  the  inks  to  assure  that  they  will  work  with  the 
print  heads.  Feeding  a  different  odorant  to  each  nozzle  would  not  only 
require  a  complete  re-engineering  of  the  via  layer,  it  would  also  require  a 
careful  study  of  the  physical  properties  of  each  odorant.  Since  none  of  the 
manufacturers  will  provide  the  components  that  are  assembled  into  the 
print  heads,  either  an  entire  print  head  would  have  to  be  fabricated  or  an 
existing  device  would  have  to  be  taken  apart  and  a  custom  fabricated  via 
layer  would  have  to  be  inserted. 

III.D.3.  Micro  Fluidics 

But  what  of  MEMEs  technology?  Surely,  microfluidic  devices  are  an  active 
area  of  research  in  this  field.  True,  but  it  is  seldom  noted  that  the  most 
economically  significant  MEMEs  technology  is  in  the  ink  jet  printer  and  so 
the  problems  described  above  have  significance  for  MEMEs  devices  in 
general. 

We  talked  to  a  number  of  the  major  laboratories  including  Livermore, 
Sandia,  Oak  Ridge,  and  Sarnoff  about  their  miniaturization  technologies. 
When  we  approached  Sarnoff  Labs,  we  were  pleased  that  they  were  already 
working  with  the  ink  jet  printer  manufacturers  to  see  how  well  the  Navier 
Stokes  equations  described  fluid  phenomena  in  their  print  heads. 

We  asked  Dr.  Anne-Marie  Alanzillato  whether  she  could  fabricate  a 
microfluidic  pump  array  that  we  could  use  for  odorant  delivery.  She  was 
very  enthusiastic  about  the  project  as  was  Ken  Gabriel  who  had  funded  her 
work  when  he  was  at  DARPA.  However,  there  were  limitations  and 
challenges.  Only  9  pumps  could  put  in  a  single  device.  This  was  because  of 
minimum  size  of  the  physical  connectors  that  could  bring  odorant  material 
into  the  chip.  Like  the  ink  jet  printer,  it  could  release  25  micron  droplets 
every  25  microseconds.  While  she  felt  that  the  corrosive  characteristics 
of  odorants  would  probably  not  be  a  problem,  the  problem  of  handling  a 
variety  of  odorants  dissolved  in  variety  of  diluents  with  a  wide  range  of 
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physical  properties  has  never  been  addressed  let  alone  solved.  Therefore, 
it  was  recognized  that  such  a  device  would  have  a  finite  risk  of  failure. 

Though  both  parties  were  very  interested  in  pursuing  this  design,  there 
were  some  concerns.  The  9  odor  limit  was  at  the  lower  limit  of  what  we 
thought  would  be  sufficient  for  real  applications,  so  the  possibility  of 
two  or  more  chips  per  systems  had  to  be  considered.  The  replication  costs 
of  the  chips  ($600)  would  not  be  unreasonable  to  support  high  end 
research,  but  it  would  not  guarantee  a  market  for  odors  in  surgical 
simulators. 

It  is  important  to  note  that  the  micropumping  element  is  not  by  itself  a 
complete  odor  delivery  system.  It  was  to  include  1  CC  reservoirs  for  the 
odorants  as  well  as  the  drive  electronics.  The  major  unknown  was  how 
fast  the  25  micron  droplets  would  vaporize.  If  the  output  of  the 
micropump  array  was  a  vapor,  it  could  be  mounted  on  the  head  and  output 
directly  into  the  wearer’s  nose.  This  would  eliminate  the  pumps,  valves, 
micro-venturi  systems,  much  tubing,  and  reduce  the  size  of  the  batteries 
that  had  to  be  carried  considerably.  On  the  other  hand,  if  the  droplets  did 
not  vaporize,  one  would  be  reluctant  to  allow  them  to  be  directly  inhaled, 
fearing  that  the  liquid  odorants  that  eat  valve  coatings  would  also  harm 
nose  and  lung  tissues.  In  this  case,  it  would  still  be  necessary  to  keep  the 
batteries,  pumps,  micro-venturis,  and  mount  the  system  on  the  back.  While 
the  micropump  array  would  be  smaller  than  one  created  with  individual 
valves,  the  total  assembly  would  only  be  fractionally  smaller  than  the  one 
we  have  designed. 

Even  with  these  concerns,  the  device  would  be  well  worth  building 
because  it  would  have  represented  an  important  step  forward  in 
approaches  to  odor  delivery.  The  show  stopper  was  the  cost  of  the  effort 
which  was  to  be  $200K  which  was  the  entire  budget  including  our  salaries 
and  overhead  for  the  third  year  of  the  program  in  which  the  final  delivery 
system  was  to  be  developed.  It  would  also  take  a  year  to  get  it  fabricated 
and  during  that  time  we  would  have  nothing  to  work  with,  which  meant 
that  we  still  had  to  develop  an  alternative  system.  Thus,  we  embarked 
upon  the  developing  the  delivery  system  described  earlier.  Ironically  some 
time  later.  Dr  Alanzillato  convinced  her  management  to  provide  half  of  the 
support  for  the  effort  from  internal  funds.  By  then,  it  was  too  late  for  us 
to  take  advantage  of  her  offer. 
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1II.D.4.  Design  Dilemma 

One  caveat  that  should  be  kept  in  mind  when  considering  a  miniature 
device  is  provided  by  careful  attention  to  the  sounds  that  an  ink  jet 
printer  makes  when  it  is  being  turned  on  or  when  a  cartridge  has  been 
changed.  Even  though  it  has  been  carefully  designed  for  the  exact  inks  that 
it  is  printing  (which  are  chemically  benign),  every  ink  jet  printer  goes 
through  minutes  of  mechanical  warmup  procedures  that  involve  spitting 
ink  onto  a  sponge  mounted  beyond  the  end  of  the  print  line  among  many 
other  mysteries.  If  such  precautions  against  clogging  are  necessary  with 
the  simplest  materials,  what  additional  problems  will  arise  with  a 
general  purpose  device  that  is  expected  to  handle  a  wide  range  of 
odorants. 

The  dilemma  for  the  design  of  the  delivery  system  is  that  it  is  circular. 
Normally,  one  would  start  with  the  requirements  and  engineer  a  device 
that  meets  the  requirements.  The  requirements  would  derive  from  past 
experience  in  the  design  domain.  These  would  include  the  formulations  of 
the  odorant  materials,  the  physical  properties  of  the  odorant  materials, 
the  sensitivity  of  the  human  nose  to  those  materials,  the  compatibility  of 
the  odorant  materials  with  those  used  in  the  delivery  device,  and  the  safe 
exposure  levels  of  the  odorant  materials. 

In  this  case,  many  of  the  odorant  materials  that  one  would  like  to  use  are 
not  yet  available.  Furthermore,  even  for  the  odorant  materials  that  we 
have  been  able  to  acquire,  the  physical  properties,  human  sensitivity  to 
the  materials,  and  their  chemical  properties  are  not  available — even  from 
the  company  that  sells  them.  (Even  something  as  basic  as  possible 
solvents  was  not  available.) 

The  only  exception  is  the  toxicity  information  which  provided  by  the 
manufacturer  for  each  of  the  individual  chemicals  although  not  for 
mixtures  made  from  them.  This  data  is  very  conservative  and  warns  that 
many  of  the  materials  are  carcinogenic  or  otherwise  harmful.  No  dosage 
level  is  specified.  Getting  the  toxicity  information  for  diluted  odorants 
required  engaging  a  toxicologist  charging  hundreds  of  dollars  an  hour  and 
she  found  the  task  frustrating  and  raised  the  cost  estimates  for  future 
work. 

The  perceptual  information  on  the  human  ability  to  detect  each  chemical 
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in  specific  concentrations  is  also  not  available.  The  only  way  to  acquire  it 
is  to  build  special  odor  delivery  instrumentation  which  has  much  more 
precise  control  of  dosage  than  is  needed  for  virtual  reality  simulations.  In 
retrospect,  such  a  device  should  have  been  contemplated.  Such  studies  are 
done  in  odor  research,  but  both  the  instrumentation  and  the  methodology 
seemed  crude  to  the  PI.  It  was  aimed  at  single  chemicals,  not  mixtures.  It 
was  very  slow  and  therefore  unsuitable  for  simulations.  It  was  based  on 
passive  evaporation  rather  forced  vaporization  which  was  deemed  better 
for  complex  mixtures..  Thus,  the  design  problem  is  that  it  is  necessary  to 
have  delivery  system  in  order  know  the  requirements  of  such  a  system. 

IV.  Odorant  Acquisition 

When  the  project  began,  we  were  working  with  Monell  Chemical  Senses 
Center  of  Philadelphia  who  assured  us  that  the  odors  needed  for  most  of 
our  simulations  would  be  easy  to  obtain.  It  was  their  responsibility  to 
identify  and  obtain  the  required  materials.  Unfortunately,  it  turned  out 
that  they  had  badly  miscalculated  and  few  of  the  materials  were  available 
off-the-shelf  and  many  had  never  been  characterized.  They  had  some 
ability  to  do  perfuming  but  seemed  to  be  very  reluctant  to  actually  do  it. 
After  a  year  and  a  half,  we  were  only  able  to  get  them  to  produce  a  very 
few  of  the  odors  we  needed;  however,  since  it  was  not  a  task  they  were 
very  interested  in  and  we  finally  felt  forced  to  seek  other  sources  of 
support  for  this  part  of  the  project. 

IV.A.  Fragrance  Companies 

The  first  step  was  to  approach  and  do  presentations  to  a  number  of  the 
large  fragrance  companies.  These  firms  found  the  project  fascinating  but 
not  commercial  from  their  point  of  view.  The  typical  question  was  how 
many  tons  of  material  would  we  need.  Each  company  provided  us  four  or 
five  odors  that  they  thought  might  be  helpful  of  which  only  one  would 
typically  be  convincing.  This  experience  showed  us  the  paucity  of 
materials  intended  to  imitate  the  odors  in  the  real  world.  While  floral 
fragrances  and  fruity  smells  are  highly  elaborated,  very  few  unpleasant 
odors  have  been  created.  These  are  often  developed  as  models  for  chemists 
to  work  against  as  they  try  to  mask  or  neutralize  the  noxious  odor  with  a 
deodorant.  Thus,  some  of  these  companies  have  developed  materials  that 
simulate  human  sweat,  baby  vomit,  and  kitty  litter.  Unfortunately,  these 
models  are  highly  proprietary  and  jealously  guarded  corporate  secrets. 
Several  individuals  contacted  had  spent  much  of  their  careers  developing 
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just  one  of  these  odors. 


One  reaction  that  these  companies  had  when  presented  with  the  list  of 
odorant  materials  that  we  were  looking  for  was  that  anything  that  was 
associated  with  smoke,  vehicle  exhaust,  and  explosives  would  involve 
materials  that  might  be  considered  hazardous  and  that  they  did  not  want 
to  incur  any  liability  for  creating  such  materials.  Indeed,  almost  everyone 
we  encountered  was  completely  spooked  by  any  mention  of  potentially 
hazardous  materials. 

IV.B.  Perfumer 

The  next  approach  we  tried  was  to  enlist  the  services  of  a  free  lance 
professional  perfumer,  Geraldine  O’Keefe.  She  was  interested  in  the 
project  and  thought  that  she  would  be  able  to  contribute.  However,  the 
term  “free  lance”  turned  out  to  be  misleading.  While  she  does  not  work  for 
any  company,  she  does  use  the  facilities  of  large  companies  to  ply  her 
trade.  If  she  has  an  idea  or  a  commission,  she  must  interest  a  fragrance 
house  in  her  project  in  order  to  get  access  to  their  resources.  Usually, 
they  are  hoping  to  produce  whatever  she  develops. 

The  resources  required  include  a  chemistry  lab  with  sophisticated 
analytical  equipment,  i.e.  a  gas  chromatograph  (GC)  and  a  mass 
spectrometer  (MS)  connected  in  series  so  the  output  of  the  GC  flows  into 
the  MS.  However,  traditional  lab  equipment  has  several  problems.  First, 
the  human  nose  is  better  than  any  single  laboratory  instrument,  so 
substances  which  smell  may  elude  detection.  Second,  many  of  the 
components  of  a  substance  may  not  contribute  to  the  smell  at  all.  Going  to 
the  trouble  to  simulate  these  materials  would  obviously  be  a  waste  of 
time  and  money. 

The  perfumer’s  role  in  the  analysis  is  to  gauge  the  importance  of 
components  that  are  not  picked  up  by  the  analysis  and  to  identify  the  odor 
properties  of  those  that  are.  A  perfumer  working  with  familiar  materials 
will  know  which  ones  are  odorous  and  which  ones  are  not.  However,  for 
unknown  materials,  it  is  valuable  to  be  able  smell  the  constituents  that 
are  separated  by  the  gas  chromatograph  before  they  are  passed  on  to  the 
mass  spectrometer.  While  this  is  usually  not  possible,  a  few  GC/MS 
systems  are  equipped  with  what  is  called  a  “Sniffer  Port”  which  allows 
the  operator  to  smell  the  ingredients  as  they  pass  from  one  device  to  the 
other.  An  alternative  design  would  be  to  have  a  stand-alone  GC  that  is 
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identical  to  the  one  used  in  a  GC/MS.  Then,  the  operator  can  smell  the 
output  of  the  GC,  characterize  the  odor,  and  record  the  time  at  which  it  is 
emitted.  When  the  same  material  is  put  through  the  GC/MS,  it  will  be 
possible  to  correlate  the  previously  recorded  smells  with  the  analysis 
provided  by  the  MS.  There  are  very  few  instrument  systems  equipped  with 
sniffer  ports  which  are  available  to  outside  users.  Even  some  of  the 
fragrance  companies  eschew  the  use  of  a  sniffer  port  because  of  concerns 
about  their  own  safety. 

Begin  CONFIDENTIAL 

IV.C.  Odor  Mixing  Technology 

One  thing  that  struck  the  PI  when  he  visited  fragrance  companies  was  that 
perfumers  formulate  odorants  by  mixing  liquids.  While  may  seem  like  an 
unremarkable  observation,  the  practice  does  mean  that  they  cannot  backup. 
Once  they  have  added  an  ingredient,  they  have  no  convenient  way  to  remove 
it.  Since  the  materials  are  expensive,  they  must  be  very  conservative  as 
they  experiment  with  new  compositions. 

It  occurred  to  the  PI  that  the  delivery  technology  being  developed  under 
this  project  offers  a  radically  different  approach.  Our  odor  delivery 
devices  are  typically  used  to  select  one  odorant  at  a  time  from  a  small 
inventory  of  ten  or  more  options.  However,  there  is  no  reason  that  the 
same  devices  cannot  release  more  than  one  odor  at  a  time.  By  mixing 
vapors  rather  than  liquids,  the  operator  has  almost  no  investment  in  the 
current  odor  and  could  in  principle  change  ingredients  and  proportions 
through  a  GUI  interface.  Indeed,  whereas  the  professional  perfumer 
proceeds  tentatively  and  deliberately,  a  vapor  mixing  system  could  modify 
the  mixture  on  its  own  under  the  direction  of  a  genetic  algorithm.  As  in 
the  graphics  work  that  Karl  Sims  did  when  he  was  at  Thinking  Machines, 
the  system  could  provide  a  set  of  alternatives,  ask  the  operator  which  one 
he  preferred,  generate  four  variations  on  the  selection,  and  repeat  the 
process  until  he  was  satisfied  or  felt  that  he  was  in  a  blind  alley.  Using 
Simm’s  system,  it  was  possible  for  non-artists  to  create  beautiful 
images.[3]  Our  thought  is  that  odors  could  be  mixed  the  same  way. 

The  limitation  of  the  delivery  technology  developed  so  far  is  that  we  do 
not  currently  have  the  large  inventory  (thousands)  of  odorants  that  a 
professional  perfumer  is  typically  surrounded  by.  However  for  any  given 
project,  these  individuals  create  a  small  working  set  of  from  20-50 


25 


bottles  and  then  start  mixing.  There  is  no  reason  that  a  version  of  our 
system  that  would  support  such  a  repertoire  could  not  be  developed. 

Of  greater  concern  would  be  the  need  to  provide  precise  control  of  the 
proportions  of  each  of  the  materials.  We  currently  provide  proportional 
control  by  pulse  width  modulation,  i.e.  we  vary  the  duty  cycle,  the 
percentage  of  the  time  a  valve  is  open  versus  the  time  it  is  closed  for 
evaporative  techniques.  With  the  droplet  based  systems,  we  vary  the 
number  of  droplets  per  unit  time.  Since  the  frequencies  that  we  were 
using  to  control  droplet  generation  are  quite  low  (300-400  hertz),  we 
thought  we  might  not  have  sufficient  dynamic  range  for  mixing  odorant 
materials. 

Finally,  there  is  the  question  of  correlating  the  number  of  droplets 
vaporized  with  the  liquid  proportions  that  a  perfumer  would  need  to 
create  the  same  odor.  Our  venturi  system  assures  that  all  ingredients  are 
vaporized  completely  and  instantaneously.  The  key  physical  property 
associated  with  perfume  ingredients,  the  vapor  pressure  or  tendency  of 
each  component  to  evaporate  is  being  finessed.  In  our  system,  every 
material  evaporates  whether  it  wants  to  or  not.  If  it  was  part  of  a 
mixture  of  many  ingredients,  the  most  volatile  components  would 
evaporate  first.  Nevertheless,  it  should  be  possible  to  work  out  the 
correspondence  between  vapor  proportions  and  liquid  proportions. 

For  virtual  reality  applications,  the  issue  does  not  matter  since  we 
display  vapors.  Therefore,  the  proposed  system  could  be  used  for  mixing 
odors  for  virtual  reality  simulations.  Even  for  perfume  companies,  the 
delivery  technology  suggests  a  new  way  of  packaging  and  presenting  fine 
fragrances.  A  wearable  fragrance  appliance  could  present  the  out-of-the- 
bottle  experience  throughout  the  day  rather  one  that  fades  the  longer  the 
perfume  is  worn. 

End  CONFIDENTIAL 

IV.D.  Odor  Chemical  Acquisition 

Given  the  poor  substitutes  for  familiar  odors  that  the  fragrance 
companies  had  shown  us,  we  despaired  of  getting  truly  accurate  odors  for 
our  simulations.  At  the  same  time,  there  is  evidence  that  people’s  sense 
of  smell  can  be  fooled.  For  instance,  there  was  a  study  in  which  people 
tasting  an  apricot  flavored  red  candy  believed  that  it  was  cherry  because 
of  the  color.  In  some  environments  where  people  do  not  have  a  very  clear 
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idea  what  to  expect  but  do  know  that  it  should  smell  somehow,  it  may  be 
enough  to  present  some  reasonable  odors  rather  than  the  correct  ones,  e.g. 
general  vegetation  odors  may  be  enough  to  give  life  and  texture  to  outdoor 
scenes. 

IV.D.1.  Direct  Purchase 

With  this  thought  in  mind,  we  acquired  a  flavors  and  fragrances  catalog 
from  Aldrich  Corporation,  but  were  surprised  to  discover  that  they  would 
not  sell  to  us.  They  explained  that  their  chemicals  were  only  sold  in 
“neat”  form,  meaning  that  they  are  pure  rather  than  diluted.  In  this  form, 
they  are  almost  all  extremely  toxic.  Since  we  were  not  a  chemistry  lab, 
they  did  not  want  the  liability  attendant  to  selling  us  such  materials. 

IV.D.2.  UCONN  Food  Sciences 

Since  we  did  not  want  to  handle  toxic  materials  or  to  be  responsible  for 
disposing  of  them,  their  position  seemed  reasonable  and  prudent.  So,  we 
approached  the  Food  Sciences  Laboratory  at  the  University  of  Connecticut 
to  see  if  they  would  purchase  the  chemicals,  dilute  them  to  safe  levels 
with  the  proper  diluents,  and  then  store  them  until  we  needed  them.  While 
the  initial  reaction  was  very  enthusiastic,  after  several  months  of 
indecision  the  university  decided  that  they  also  were  sufficiently 
concerned  about  liability  issues  that  they  were  be  afraid  to  become 
involved. 

IV.D.3.  Environmental  Inspectors 

Throughout  the  project  we  have  been  talking  with  a  small  consulting  firm 
in  Bloomfield  Connecticut  called  Odor  Science  and  Engineering,  Inc.  They 
primarily  work  with  odors  as  nuisances  or  as  evidence  of  pollutants.  There 
are  a  variety  of  federal,  state,  and  local  regulations  that  control 
emissions  from  sewage  treatment  facilities,  chemical  plants,  meat 
rendering  plants,  hog  farms,  etc.  As  a  consequence,  governmental 
regulators  and  the  operators  of  some  of  these  facilities  may  enlist  this 
firm  to  determine  where  the  odor  that  has  given  rise  to  a  particular 
complaint  is  coming  from.  This  focus  on  environmental  odors  means  that 
this  firm  and  others  like  it  have  expertise  about  a  range  of  unpleasant 
odors  that  fragrance  companies  have  never  had  any  reason  to  think  about, 
such  as  those  emanating  from  hog  farms. 

Of  particular  interest  to  us  is  the  fact  that  they  routinely  take  samples  of 
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noxious  air  that  has  given  rise  to  complaints  and  have  it  analyzed.  Since 
like  perfumers  they  are  as  interested  in  the  smell  as  in  the  chemistry, 
they  do  need  a  sniffer  port  to  let  them  know  which  ingredients  of  a  GC/MS 
analysis  are  causing  the  smell.  The  closest  one  that  they  know  of  is  in 
California  which  means  that  they  must  fly  out  to  the  West  Coast  each 
time  they  want  to  analyze  a  sample.  By  manning  the  sniffer  port 
themselves,  they  absolve  the  testing  firm  of  liability  and  they  assure 
consistency  of  their  description  of  odor  properties.  (One  thing  that  has 
amazed  us  is  how  almost  every  individual  In  the  industry  seems  to  use 
entirely  different  adjectives  to  describe  the  odors  we  have  shown  them. 
They  all  draw  on  the  same  set  of  descriptors,  but  use  them  completely 
differently.  This  means  that  they  have  limited  ability  to  communicate 
with  each  other.  It  would  be  useful  to  try  to  quantify  these  attributes.) 

Getting  air  samples  analyzed  requires  the  cost  of  the  analysis  plus  the 
cost  of  flying  the  head  of  Odor  Science  and  Engineering  out  to  California 
plus  compensating  him  for  the  service  (at  $175  per  hour).  Since  he  can 
only  analyze  three  odors  per  day,  this  is  obviously  an  expensive 
proposition. 

The  is  another  difficulty.  Air  samples  must  be  captured  in  metallized  bags 
with  special  valves.  There  is  concern  about  some  of  the  components  of  the 
odors  adhering  to  the  insides  of  the  bags.  Even  more  difficult  is  the  fact 
that  the  samples  must  be  analyzed  within  24  hours  after  they  are 
collected.  Since  some  of  our  odors  must  be  collected  in  different  locations 
under  different  conditions,  this  time  pressure  makes  what  would  seem  to 
be  a  routine  laboratory  analysis  seem  more  like  a  special  forces  operation. 

Given  an  analysis  with  a  list  of  chemical  components  and  a  subjective 
judgment  of  each’s  contribution  to  the  overall  odor  experience,  there  are 
several  additional  hurdles: 

1.  an  identified  material  may  be  toxic  and  a  similar  smelling  substitute 
must  be  found. 

2.  an  identified  material  may  not  be  available  commercially  even 
though  the  means  of  producing  it  are  known 

3.  an  identified  substance  may  not  have  been  studied  scientifically  in 
the  past  in  which  case  a  synthetic  chemistry  lab  will  need  to  be  enlisted 
to  crystallize  it,  determine  its  molecular  structure  by  x-ray 
crystallography,  and  synthesize  an  analog.  This  process  can  be  very 
expensive.  Monell  has  had  grants  of  hundreds  of  thousands  of  dollars  to 
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analyze  a  single  natural  odor. 


IV.D.4.  Toxicologist 

Odor  Science  and  Engineering  has  been  helpful  to  us  in  a  number  of  ways 
having  to  do  with  the  ingredients  of  noxious  materials.  Initially,  they 
were  willing  to  purchase  chemicals  for  us,  but  upon  reflection  decided 
that  they  were  no  more  equipped  than  we  to  do  the  handle  the  chemicals. 
So,  they  referred  us  to  another  firm,  TRC  Environmental,  Inc  from  which 
they  had  once  spun  off. 

TRC  is  engaged  in  similar  activities  to  Odor  Science  and  Engineering.  In 
addition,  they  have  a  complete  chemistry  and  toxicology  lab.  Dr  Karen 
Vetrano  is  a  toxicologist  and  has  been  enlisted  to  acquire  odorant 
materials  for  us.  Her  tasks  are  to: 

1.  determine  the  toxicology  of  the  substances  we  are  interested  in 

2.  determine  the  physical  properties  of  each  substance  such  as  vapor 
pressure,  surface  tension,  viscosity,  etc  to  the  extent  that  these  are 
available  through  the  literature. 

3.  determine  what  solvents  can  be  used  to  dissolve  the  materials 

4.  determine  what  concentrations  are  safe  for  us  to  handle  and  the 
strongest  of  these  that  would  be  useful  for  odor  display.  We  will  dilute 
down  from  these  levels  ourselves. 

5.  actually  order  the  materials  and  prepare  dilutions  of  them  for  us 

6.  research  the  availability  of  analyses  and  synthetic  formulations  for 
a  number  of  odors  that  would  be  used  to  represent  military  emergency 
situations  such  as  gasoline,  diesel  fuel,  exhaust,  explosives,  burning 
electrical  insulation,  etc.  While  some  of  these  formulations  have  been 
identified,  none  of  them  are  available  as  mixtures  available  for  purchase. 
There  is  also  some  question  as  to  whether  they  smell  like  the  real  odor 
they  are  intended  to  simulate. 

The  above  process  has  been  pursued  for  a  suite  of  30  odors  from  a  list  of 
180  that  we  had  identified  as  being  of  interest  to  us.  (Since  the  Aldrich 
catalog  only  has  a  few  descriptors  for  each  odorant,  we  can  only  smell 
them  after  we  have  had  them  prepared  for  us.)  Only  a  couple  of  these 
odorants  will  are  immediately  identifiable.  Others  have  qualities  that 
suggest  smoke  or  vegetation,  but  are  only  components  from  which 
representational  odors  would  be  created.  This  was  an  expensive  process 
and  one  whose  cost  was  revised  upwards  by  TRC  for  future  orders. 
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IV.D.5.  Cornell  Food  Flavor  Laboratory 

Another  odor  search  connection  has  been  an  effort  with  the  Food  Flavor 
Laboratory  at  Cornell  University  operated  by  Dr.  Terry  Acree.  This 
laboratory  does  do  analysis  and  synthesis  of  organic  flavor  and  odor 
material  for  the  food  industry.  They  have  the  analytical  instruments  to 
identify  the  components  in  an  unknown  odor.  They  also  have  experience  of 
making  simulated  materials.  While  their  GC/MS  machine  does  not  have  a 
sniffer  port,  they  do  have  a  stand-alone  GC  identical  to  the  one  in  their 
GC/MS  and  they  time  the  output  of  each  odor  from  the  GC  before  putting 
the  same  sample  in  the  GC/MS.  They  are  very  concerned  about  any  hint  of 
hazardous  materials  and  are  unwilling  to  ask  their  students  to  sniff  them. 
However,  for  substances  which  are  known  to  be  safe  because  they  are 
naturally  occurring  organics  such  as  the  smell  of  blood,  this  group  might 
be  of  doing  the  analysis/synthesis  required. 

Their  funding  is  almost  entirely  related  to  food.  In  this  area,  their 
expertise  in  fruity  tastes  and  smells.  Vegetables  have  not  been 
investigated  very  much.  Meats  and  therefore  tissues  have  not  been 
investigated  at  all.  The  odors  associated  with  accident  and  battlefield 
scenarios  will  never  be  addressed.  (The  continuum  from  floral  essences  in 
the  fragrance  industry  to  fruits,  vegetables,  meats,  and  combustion  odors 
corresponds  to  a  recognized  spectrum  of  difficulty  in  odor  development. 
The  pleasant  odors  which  are  easy  to  replicate  are  made  up  of  components 
with  low  molecular  weight.  Therefore,  many  of  the  odors  needed  for 
medical  simulation  are  inherently  more  difficult  to  simulate  as  well  as 
less  lucrative.) 

Their  lab  costs  are  very  high.  Having  Dick  Duffee  of  Odor  Science  and 
Engineering  fly  out  to  California  to  use  the  equipment  there  was  no  more 
expensive.  After  many  efforts  to  interest  them  in  what  we  were  doing,  we 
rented  their  team  for  a  day  to  brain  storm  what  they  could  do  to  simulate 
a  real  world  situation  based  on  their  existing  expertise.  The  result  which 
is  included  as  Appendix  B  uses  individual  chemicals  to  simulate  each  odor 
in  their  scenario.  The  chemicals  identified  contained  no  examples  that  the 
PI  had  not  already  identified  from  the  Aldrich  catalog  without  being  able 
to  smell  them. 

IV.D.6.  Turk  Test 

After  many  calls  to  organizations  that  would  seem  to  have  an  interest  in 
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unpleasant  odors,  e.g.  the  Environmental  Protection  Agency,  the  American 
Petroleum  Institute,  and  a  number  of  environmental  monitoring  firms  in 
the  private  sector,  we  located  the  formulation  for  the  Turk  Test  which 
was  used  for  testing  the  effects  of  diesel  exhaust  odors  on  human 

subjects  in  the  late  1960s. [1]  Later,  as  we  were  trying  to  get  the 
chemicals  used  in  the  formulation,  we  found  someone  who  knew  how  to 
get  in  contact  with  Amos  Turk  who  is  now  80  years  old. 

The  PI  called  him  and  has  had  a  number  of  conversations  about  odor 
display.  He  was  quick  to  point  out  that  his  simulated  diesel  odor  did  not 
smell  like  diesel  fuel.  He  explained  that  what  we  think  of  as  the  smell  of 
diesel  exhaust  is  not  exactly  a  smell  at  all.  It  contains  droplets  of 
unburned  fuel,  particulate  matter  from  the  combustion  process,  and 
fugitive  combustion  products  which  are  in  the  process  of  turning  into 
something  else.  It  also  contains  benzene  which  is  a  known  carcinogen. 
Therefore  what  he  had  done  was  to  simulate  the  qualities  of  the  diesel 
odor  with  completely  different  materials  which  were  known  to  be  safer 
but  which  did  not  necessarily  smell  similar.  These  qualities  included 
burnt,  oily,  pungent,  and  aldehydic.[2]  This  information  leads  to  the 
paradox  that  the  only  simulation  of  a  fuel  odor  does  not  like  it.  Since  it 
would  cost  thousands  of  dollars  to  get  the  Turk  Test  formulation  made  up 
for  us,  we  are  reluctant  to  pursue  it  if  it  does  not  smell  like  the  real 
thing. 

At  the  same  time,  the  Turk  Test  points  to  an  alternative  approach  to  odor 
simulation,  the  representation  of  qualities  rather  than  the  odors 
themselves.  (Dr.  Turk’s  report  on  the  techniques  available  for  simulating 
is  included  as  Appendix  C.)  It  also  points  to  the  need  for  broadening  the 
requirements  on  odor  delivery  beyond  the  smells  themselves  to  include 
droplets  and  particles.  It  is  possible  that  water  droplets  and  dust 
particles  of  some  benign  substance  could  represent  those  aspects  of  real 
odors  that  are  usually  caused  by  more  noxious  materials.  One  example  of 
where  this  would  be  useful  is  in  urban  warfare  training  simulations  where 
soldiers  are  trained  to  clear  buildings  of  enemy  personnel  by  using  their 
weapons  to  cut  doorways  through  walls.  Clearly,  this  practice  would  lead 
to  large  quantities  of  dust  in  the  air  which  would  should  be  represented  in 
the  training. 

While  we  will  continue  to  pursue  additional  sources  of  odorant  materials. 
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we  are  confident  that  our  experience  so  far  represents  the  reality  of  odor 
simulation  and  that  we  do  not  have  the  resources  or  the  willing  partners 
to  cause  the  desired  odors  to  be  created.  Instead,  we  will  have  to  work 
around  the  odors  that  we  cannot  acquire  and  do  the  best  we  can  with  what 
we  have. 

V.  Motion  Tracking  for  Odor  Delivery  During  Natural  Locomotion 

Many  of  the  people  working  in  virtual  reality  have  by  now  been  convinced 
by  the  Pi’s  arguments  of  the  past  decade  claiming  that  current  HMD 
technology  is  inadequate  for  almost  any  purpose.  His  repeated  question  in 
over  one  hundred  talks  and  in  his  book  was,  “Would  you  use  it,  if  it  was 
free?”  Since  even  the  most  committed  HMD  aficionados  had  to  ruefully 
admit  that  they  would  not  and  in  their  own  labs  did  not  use  HMDs  for  any 
length  of  time. 

However,  the  PI  was  always  quick  to  point  out  that  the  ideal  HMD  would 
have  some  powerful  advantages  over  any  other  kind  of  display.  Most 
important  of  these  would  be  the  ability  to  move  around  a  virtual  world  by 
walking  naturally  rather  than  by  some  clumsy  control  convention  such  as 
pointing  your  finger  in  the  direction  you  want  your  eyes  to  fly.  In  this 
project,  we  have  already  seen  that  coupling  odor  display  to  natural  head 
movements  is  very  compelling.  We  have  always  believed  that  odors 
encountered  while  walking  about  the  virtual  world  would  seem  even  more 
so.  Natural  movement  is  impossible  with  the  very  few  systems  that 
permit  movement,  because  the  participant’s  head  is  connected  to  the 
computer  by  the  wires  that  connect  the  tracking  system  to  the  sensors  he 
is  wearing.  Therefore,  a  wireless  tracking  system  is  needed  for  testing 
odor  display  in  immersive  virtual  reality. 

V.A.  Preliminary  Tracking  System 

In  the  first  year  of  the  project,  a  preliminary  wireless  tracking  system 
was  developed  that  had  performance  similar  to  that  of  the  early  magnetic 
trackers.  This  system  used  overhead  video  cameras  looking  down  at  a 
multi-LED  target  worn  on  the  top  of  the  participant’s  head.  It  analyzed  the 
pattern  and  determined  the  location  and  orientation  of  the  head  from  that 
information.  Because  it  was  using  standard  video  cameras,  it  was  limited 
to  30  samples  per  second.  Its  spatial  accuracy  was  also  limited  for  the 
same  reason.  The  tracking  worked  as  did  the  hand  off  from  one  camera  to 
the  next  down  the  40’  path.  Only  the  failure  of  the  wireless  video 
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transmission  prevented  this  from  being  an  interesting  system.  Since  then, 
the  video  transmission  problem  has  been  solved. 

V.B.  Tracking  Design  Alternatives 

Based  on  the  lessons  of  that  system,  a  number  of  much  more  sophisticated 
ways  of  head  tracking  have  been  considered.  These  include: 

1.  head-worn  laser  diodes  that  would  project  spots  on  the  floor  walls 
and  ceiling.  The  locations  of  these  spots  would  be  analyzed  to  find  the 
tracking  information. 

2.  laser  measurement  devices  which  have  been  getting  very  accurate. 

3.  scanning  the  environment  with  a  sheet  of  laser  light.  The  scan  angle 
at  the  moment  the  laser  was  detected  by  a  sensor  on  the  participant’s 
body  would  determine  its  location  in  one  dimension.  Alternate  scanning  in 
X  and  Y  would  provide  two-dimensional  information.  A  second  scanning 
system  would  provide  the  data  for  stereo  three-dimensional  calculations. 

4.  high-speed,  high-resolution  video  cameras  to  increase  the  spatial 
and  temporal  resolution  of  the  overhead  video  tracking  system. 

5.  special  cameras  which  can  increase  their  scan  rate  by  aborting  a 
scan  when  the  target  is  found  or  can  skip  over  lines  in  which  the  target  is 
unlikely  to  be  found. 

6.  a  special  camera  which  had  onboard  processing  that  would  allow  it 
to  in  one  operation  test  an  entire  line  to  see  if  it  was  empty. 

7.  inertial  and  ultrasonic  techniques  which  have  since  been  developed 
as  a  product  by  InterSense.  (Systron  Donner  was  prepared  to  make  an 
inertial  system  for  the  PI  in  1994,  but  delays  in  the  DARPA  funding  and 
the  size  of  the  resulting  assembly  caused  that  approach  to  be  set  aside.) 
The  performance  of  the  InterSense  system  is  good  as  long  as  it  is 
tethered.  Wireless  operation  has  been  reported  using  an  array  of 
ultrasonic  transducers. [4] 

8.  position  sensing  diodes  which  are  very  fast  detectors  and  provide  the 
X-Y  centroid  of  a  spot  of  light  that  falls  upon  them.  A  year  ago,  the 
University  of  North  Carolina  revealed  that  they  are  using  these  devices. 
Their  system  is  fast  and  accurate,  but  it  still  tethered  and  will  slow  down 
if  wireless  transmission  is  attempted.  The  PI  has  been  interested  in  these 
devices  since  the  early  1970s  and  always  shied  away  from  them  because 
of  the  analog  nature  and  limited  resolution.  Consequently,  he  is  very 
impressed  with  UNC’s  achievement. 

9.  a  magnetic  system  commercialized  by  Ascension. 
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Each  of  the  above  systems  had  some  disadvantages.  The  camera-based 
systems  suffered  from  the  low  resolution  of  the  video  sensors.  High 
temporal  resolution  is  possible  but  very  expensive  and  is  limited  to 
800x480  pixel  spatial  resolution.  1Kx1K  is  possible,  but  only  the  expense 
of  speed-no  more  than  30  samples  per  second.  Higher  resolution  devices 
exist,  but  they  are  even  slower.  The  other  systems  that  we  have 
considered  or  that  others  have  developed  are  inside-out  systems  as 
opposed  to  outside-in  systems.  This  means  that  the  sensors  are  worn  on 
the  participant  and  the  data  has  to  be  transmitted  back  to  the  computer. 
While  raw  sensor  signals  can  be  sent  over  wires  or  fiber  optics  at  high 
speed,  wireless  transmission  requires  that  the  data  be  sent  back  as  a 
serial  bit  stream  which  is  much  slower. 

The  PI  has  designed  two  outside-in  systems  in  which  optical  sensors 
around  the  environment  measure  the  position  of  targets  worn  on  the 
participant.  Both  of  these  systems  seek  to  provide  very  high  spatial  and 
temporal  accuracy. 


V.C.  Line  Scan  Approach 


The  first  approach  smashes  the  resolution  constraint  suffered  by  most 
optical  sensors.  Whereas  existing  video  sensors  max  out  at  about  1Kx1K, 
this  approach  uses  pairs  of  line  scan  sensors  to  create  a  virtual  6Kx6K 
camera  which  scans  at  10Khz.  Since  multiple  LEDs  comprise  the  target  and 
each  LED  is  flashed  separately,  the  actual  rate  is  somewhat  slower— about 
2.5khz. 

When  all  the  target  LEDs  are  illuminated  at  once,  determining  which  spot 
seen  by  the  sensor  corresponds  to  which  LED  from  one  scan  to  the  next  is  a 
complex  and  error  prone  calculation.  The  new  technique  hinges  on  the  fact 
that  only  a  single  LED  is  being  illuminated  at  a  time,  eliminating  the 
correspondence  calculation.  A  traditional  video  camera  cannot  take 
advantage  of  this  simple  target,  because  it  must  always  read  out  every 
pixel  in  the  scene  to  find  the  one  that  is  illuminated. 

V.C.1.  Line  Scan  Concept 

By  using  two  line  scan  sensors  mounted  orthogonally,  we  only  need  to  do 
one-dimensional  scanning.  Thus,  we  can  scan  the  6000  pixels  of  the  line 
scan  sensor  much  faster  than  we  could  scan  the  1,000,000  pixels  of  a 
high-resolution  video  camera.  By  mounting  one  of  these  sensor  pairs  in 
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each  corner  of  the  room,  we  get  complete  coverage  of  the  space  for  stereo 
measurements  of  each  of  the  sensors  regardless  of  which  way  the 
participant  is  facing.  The  very  high  sampling  rate  assures  that  the  person 
will  have  moved  a  very  small  distance  since  the  last  sample. 

V.C.2.  Line  Scan  Optics 

Ordinarily,  a  line  scan  camera  can  only  look  at  a  line  in  the  scene  at  which 
it  is  aimed.  Special  optics  have  been  designed  which  collapse  a  two- 
dimensional  camera  view  of  a  scene  onto  a  one-dimensional  sensor.  When 
the  PI  first  started  thinking  about  using  line  scan  sensors  about  a  decade 
ago,  he  was  convinced  that  the  optical  problem  was  solvable,  but  did  not 
have  the  optical  knowledge  or  resources  to  pursue  it.  This  time  having 
fully  considered  many  alternatives,  he  was  convinced  that  the  approach 
was  the  best  option  and  should  be  implemented. 

Since  the  optical  problem  was  the  unknown,  he  looked  to  outside  optical 
designers  to  look  at  the  problem  from  the  standpoint  of  feasibility.  Bob 
Perry  of  Illumination  Technologies  which  does  lighting  and  optical 
systems  for  machine  vision  applications  was  the  first  to  tackle  the 
problem.  He  decided  that  the  problem  was  tractable  but  beyond  his  level  of 
expertise.  Next,  Dr  Gary  Grimes  at  the  University  of  Alabama  agreed  to 
address  it.  Just  when  an  approach  had  been  identified  and  the  top  student 
allocated  to  it,  that  student  received  a  dream  job  offer  and  left  school. 
There  was  no  one  else  available  with  the  requisite  knowledge. 

Finally,  the  problem  was  discussed  with  Dr.  Jannick  Holland  of  CREOL  (the 
Center  for  Research  and  Education  in  Optics  and  Lasers)  at  the  University 
of  Central  Florida.  She  was  Intrigued  by  the  challenge  and  spent 
considerable  time  analyzing  it  before  coming  up  with  an  approach  that  she 
was  sure  could  be  made  to  work.[Appendix  D] 

V.C.3.  Anamorphic  Optics 

At  this  point,  the  PI  made  an  insightful  but  near  fatal  observation.  Since 
the  cameras  were  viewing  the  scene  from  the  corner,  they  needed  a  90° 
field  of  view  in  order  to  provide  complete  coverage.  However,  while  the 
90°  of  horizontal  information  was  all  useful,  the  90°  in  the  vertical 
direction  was  not.  Two  thirds  of  that  90°  would  be  looking  at  the  ceiling 
or  the  floor,  whereas  the  participant’s  head  would  be  in  a  narrow  vertical 
band  no  wider  than  30°.  Therefore,  it  would  be  desirable  if  the  optics 
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could  asymmetric,  i.e.  providing  greater  magnification  in  one  direction 
than  the  other.  Such  anamorphic  optics  are  much  more  complex  to  design 
and  there  is  almost  nothing  in  the  literature  about  such  systems  that  also 
have  such  a  wide  field  of  view.  At  this  point,  the  PI  got  greedy  and  opted 
for  the  more  complex  design,  even  though  it  would  involve  more  lenses. 
Neither  he  nor  Dr  Rolland  realized  how  much  more  complex  it  would  turn 
out  to  be. 

An  anamorphic  fisheye  has  been  designed  in  detail.  Tolerancing,  optical 
distortion,  and  color  aberration  have  all  been  considered.  (Color  aberration 
must  be  considered  because  the  IR  LEDs  have  quite  a  broad  spectrum.)  This 
first  stage  converts  a  wide  angle  rectangle  onto  a  square  image.  A  second 
stage  which  is  also  anamorphic  compresses  this  two-dimensional  image 
onto  a  line  using  a  series  of  cylindrical  lenses.  The  design  is  completely 
designed  and  ready  to  build.  A  paper  has  been  published  on  the 
design. [Appendix  E] 

As  we  proceeded  to  consider  fabrication,  the  size  of  the  lenses  required 
was  seen  to  be  a  matter  of  concern.  Dr.  Rolland  has  many  years  of 
experience  with  designing  and  building  complex  optical  systems,  but  nPt 
with  such  large  lenses.  The  line  scan  sensor  has  6K  10-micron  pixels 
which  means  that  it  is  6  cm  or  2.4  inches  long,  implying  that  the  first 
lens  of  the  fisheye  must  be  7  inches  in  diameter  with  is  very  large.  The 
cost  of  grinding  a  lens  goes  up  with  the  square  of  its  diameter  and  the 
cost  of  the  glass  itself  goes  up  with  the  cube  of  its  diameter.  As  a 
consequence,  the  total  cost  approached  $40K  for  each  of  the  eight  required 
assemblies  for  a  total  of  $320K  for  just  optical  system  before  adding  in 
the  cost  of  the  electronics  and  computer  processing.  Such  an  expense 
would  have  distorted  the  rest  of  the  project  and  would  result  in  an 
approach  that  would  be  far  too  expensive  for  others  to  emulate. 

V.C.4.  Line  Scan  Compromises 

As  a  result,  a  number  of  compromises  were  examined  to  try  to  bring  down 
the  cost  of  the  design.  Several  approaches  were  taken  including: 

1.  beating  on  the  existing  design  with  the  existing  sensor  by  using 
fewer  lenses  and  cheaper  glass.  This  was  done  and  reduced  the  cost 
considerably,  but  still  not  to  the  point  of  practicality. 

2.  reducing  the  size  of  the  lenses  by  finding  smaller  CCD  line  scan 
sensors.  This  was  accomplished  by  finding  a  device  with  7  micron  as 
opposed  to  10  micron  pixels,  resulting  in  a  4.2  cm  (1.65  inch)  sensor. 
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3.  eliminating  the  second  stage  by  performing  its  function  by  some 
other  means. 

A  number  of  ways  of  accomplishing  #3  were  considered.  From  the 
beginning,  the  PI  has  chafed  at  the  fact  that  the  second  stage,  the 
conversion  of  a  two-dimensional  Image  into  a  line  had  to  be  done  using 
optics  at  all.  Basically,  what  is  desired  is  to  take  all  of  the  pixels  in  a 
vertical  column  of  the  two-dimensional  image  and  to  sum  them  together 
to  produce  a  single  output. 

V.C.4.a.  Flat  Fiber  Option 

One  scheme  that  has  been  investigated  is  to  do  extend  the  idea  of  an 
optical  fiber  into  two  dimensions.  An  optical  fiber  is  a  thin  strand  of 
glass  which  is  covered  with  a  cladding  material  which  assures  that  light 
that  would  otherwise  leak  out  along  Its  length  is  reflected  back  into  it. 
Devices  exist  which  are  basically  solid  blocks  of  fiber  bundles  which  are 
then  tapered  as  a  unit  to  perform  magnification  functions.  Unfortunately, 
only  very  small  magnifications  are  possible  with  this  technique  with 
current  technology.  The  PI  posed  the  question  of  whether  very  thin  flat 
sheets  of  glass  could  be  covered  with  the  cladding  material  and  many 
layers  of  these  sheets  formed  together.  If  these  sheets  formed  a  very 
narrow  rectangle  in  cross  section  at  one  and  tapered  down  into  a  point  at 
the  other,  internal  reflection  would  take  light  entering  anywhere  in  the 
rectangle  and  reflect  it  to  the  point  at  the  other  end  where  it  would  be  fed 
into  a  single  pixel  sensor.  While  this  idea  would  work  in  principle,  none  of 
the  companies  approached  about  it  thought  that  they  could  fabricate  such 
a  device  with  any  process  they  already  had  operating. 

V.C.4.b.  Slit  Scan  Option 

Another  variation  was  to  have  a  single  vertical  slit  that  would  allow  light 
from  a  column  in  the  two-dimensional  Image  to  go  through  it.  All  the  light 

through  the  slit  could  be  focussed  onto  a  single  point  detector  by  a  simple 

parabolic  mirror.  The  slit  could  then  be  made  to  scan  across  the  two- 
dimensional  image  at  very  high  speed.  Several  means  of  accomplishing 
this  scanning  were  considered.  The  most  conventional  was  to  use  a 
facetted  mirror  wheel  which  would  sweep  the  image  of  the  scene  across 
the  slit  as  the  wheel  rotated.  The  problems  with  this  approach  were  partly 

the  high  cost  of  the  motors  and  mirror  wheels  which  were  not  in  stock, 

but  also  the  mechanical  aberrations  that  the  moving  parts  would  exhibit. 
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Another  issue  was  the  very  high  speed  of  the  single  optical  detector  and 
accompanying  analog-to-digital  converter. 

V.C.4.C.  Spatial  Light  Modulator 

Another  device  was  a  spatial  light  modulator  which  is  an  LCD  device  that 
reflects  light  back  from  displayed  pixels  rather  than  by  transmitting  it 
through  them  as  a  conventional  LCD  does.  In  this  case,  each  column  in  a 
two-dimensional  array  would  be  a  single  pixel.  When  selected,  all  the 
light  falling  on  that  column  would  be  reflected  out  where  it  could  be 
detected  by  a  simple  but  very  fast  single-point  photodetector.  The 
problem  with  this  approach  is  about  $250K  and  one  year  front  end  to 
create  the  custom  chip  and  $10K  per  device  after  that. 

Another  approach  is  to  fabricate  a  special  two-dimensional  sensor  with 
all  of  the  pixels  in  each  column  wired  to  each  other.  Using  seven  micron 
pixel  widths,  it  would  have  to  be  42  mm  wide  and  14  mm  high.  Seven 

micron  pixels  are  the  smallest  that  are  commonly  available,  but  it  is  not 

clear  whether  is  a  technological  constraint  or  simply  that  there  is  a  lack 
of  pressure  to  make  smaller  devices. 

If  smaller  detectors  are  developed,  the  original  anamorphic  optics  design 
becomes  more  affordable.  With  current  sensors,  it  is  necessary  to  either 

use  a  symmetrical  as  opposed  to  anamorphic  fish  eye  resulting  in  a  2/3 

loss  of  vertical  resolution  or  to  use  smaller  and  therefore  lower 
resolution  sensors  which  has  the  same  effect.  Alternatively,  the  same 
resolution  can  be  maintained  by  decreasing  the  working  area  from  the 
original  40’X40’. 

V.C.4.d.  Retroreflective  Element 

As  the  approaches  that  the  PI  developed  were  discarded  one  by  one.  Dr. 
Rolland  had  an  idea  for  an  alternative  means  of  collapsing  a  two- 
dimensional  image  onto  a  single  line  as  required  by  the  second  stage  of 
the  optical  design  for  the  line  scan  system.  Her  idea  would  eliminate  five 
custom  lens  and  replace  them  with  a  single  off-the-shelf  lens  and  an 
unconventional  optical  component.  The  crucial  component  is  a 
retroreflective  screen  manufactured  by  3M.  A  design  study  was  conducted 
at  the  University  of  Central  Florida  over  the  summer  and  a  prototype 
device  constructed.  Unfortunately,  the  properties  of  the  retroreflective 
sheet  were  far  from  ideal  and  the  LED  illuminated  a  large  area  on  the  line 
scan  sensor  not  a  point. 
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Subsequent  discussions  with  3M  revealed  that  the  poor  performance  of  the 
retroreflective  sheet  was  not  due  to  poor  manufacturing,  but  had  been 
deliberately  injected  into  the  product  in  response  to  customer  feedback 
indicating  that  users  did  not  want  theoretically  perfect  retroreflection. 
As  a  result,  3M  has  agreed  to  make  experimental  sheets  with  much  more 
controlled  retroreflective  properties  and  Dr.  Rolland  has  been  ask  by  3M  to 
characterize  the  new  sheets.  If  this  approach  were  to  pan  out,  it  would 
resurrect  the  moribund  line  scan  approach  by  greatly  reducing  the  cost  of 
the  second  stage  of  the  optics.  (The  cost  of  the  first  stage  would  be 
reduced  by  cutting  the  resolution  to  4K  by  4K  using  sensors  having  7 
micron  detectors.  The  resulting  28  mm  symmetrical  wide  angle  lenses 
could  be  off-the-shelf  and  the  distortions  corrected  for  in  software.) 


V.D.  Random-Access  Camera  Option 

Another  approach  that  the  PI  has  desired  for  many  years  and  which  only 
existed  in  the  highest  cost  systems  has  recently  become  available.  As 
mentioned  earlier,  a  video  camera  looking  at  a  spot  of  light  must  read  out 
the  entire  image  to  find  that  spot.  Even  a  line  scan  sensor  must  read  out 
the  entire  line  to  find  the  spot,  it  would  seem  much  simpler  and  much 
faster  if  it  were  possible  to  only  read  out  the  points  near  where  the  spot 
was  last  seen  to  determine  where  it  had  moved  to  since  the  last  sample. 
CCD  cameras  cannot  do  this  because  the  charge  received  for  each  pixel 
must  be  shifted  out  pixel  by  pixel  before  it  can  be  read. 


However,  a  new  CMOS  camera  can  randomly  access  any  pixel  at  any  time. 
That  means  that  if  the  last  location  of  the  target  is  known,  all  that  is 
needed  is  to  search  the  pixels  around  that  location  because  the  target 
cannot  move  very  far  between  samples.  The  faster  this  can  be  done,  the 
fewer  pixels  need  to  be  checked.  As  a  result,  a  10x10  pixel  window  around 
a  given  location  can  be  read  out  in  40  microseconds  or  25,000  times  per 
second  which  provides  a  greater  temporal  resolution  than  any  other 
method.  The  spatial  resolution  is  not  quite  so  impressive.  2Kx2K  sensors 
with  this  kind  of  operation  are  now  available  which  would  provide  sub¬ 
millimeter  accuracy  over  a  distance  of  15’x15’.  This  working  volume  is 
typical  of  many  of  the  techniques  that  have  been  evaluated  and  the  spatial 
and  temporal  accuracy  is  better. 
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V.D.1.  Low-Resolution  Random  Access 

To  test  this  approach,  we  have  acquired  a  smaller  version  of  this  sensor  in 
a  512x512  format  which  shrinks  the  working  volume  much  farther. 
However,  the  smaller  device  will  allow  use  to  make  the  whole  system 
wireless  over  a  small  area  and  allow  us  to  test  how  valuable  the  higher 
resolution  will  be.  Since  the  temporal  resolution  as  well  as  the  spatial 
resolution  is  high,  it  may  be  that  averaging  multiple  successive  very  high 
speed  samples  will  provide  stable  enough  values  for  our  purpose.  In  an 
immersive  virtual  reality  system,  stability  is  more  important  than 
absolute  resolution  because  the  participant  does  not  know  where  he 
actually  is.  Also,  the  30  hz  video  frame  rate  dictated  by  the  computer 
graphics  coupled  to  NTSC  wireless  transmission  is  really  the  bottleneck 
in  our  virtual  reality  system.  The  rest  of  the  system  cannot  go  faster  than 
that. 

However,  it  was  hoped  that  the  tracker  would  also  be  useful  for 
augmented  reality  and  for  fundamental  human  perception  studies  like 
tracking  a  person’s  eyes  as  they  move  around  a  real  environment  and  then 
comparing  their  eyes  movements  as  they  move  through  a  virtual  reality 
simulation  of  the  same  space.  If  the  two  behaviors  are  identical,  then  the 
VR  display  has  attained  a  measurable  success.  If  on  the  other  hand,  the 
visual  behavior  with  the  VR  display  contains  artifacts  that  are  not 
present  in  the  natural  visual  behavior,  then  it  is  reasonable  to  assume 
that  the  sense  of  presence  may  be  affected  and  that  the  VR  display  needs 
to  be  improved,  in  addition,  a  special  low-resolution  graphics  system 
could  be  designed  with  faster  frame  rates  and  controlled  by  a  portable 
computer  carried  by  the  participant.  This  setup  would  allow  us  to 
investigate  the  benefits  of  faster  graphics  and  faster  tracking. 

At  the  moment  we  are  building  the  512x512  system  with  the  random- 
access  CMOS  cameras.  A  single  pair  of  cameras  are  tracking  the  multi-LED 
target  at  the  speeds  needed  for  tracking  head  movement.  We  are  currently 
addressing  the  registration  issues  because  small  errors  in  the  position  or 
orientation  of  the  cameras  can  lead  to  apparent  erroneous  rotations  of  the 
head. 

V.D.2.  Ceiling-Mounted  Random  Access  Cameras 

in  addition,  the  random-access  cameras  are  compatible  with  the  original 
overhead  camera  tracking  technique  that  implemented  early  in  the 
program.  Remember  that  the  only  problem  with  the  performance  of  that 
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approach  was  the  slow  speed  (30  fps)  and  the  fact  that  multiple  samples 
of  the  orientation  angles  had  to  be  averaged  to  produce  completely  stable 
results.  Since  the  random-access  camera  can  locate  an  LED  in  a 
millisecond,  the  original  algorithms  will  produce  stable  angles  at  a  speed 
consistent  with  the  constraint  imposed  by  the  graphics  speed.  In  addition, 
the  512x512  resolution  is  twice  that  of  the  original  system  which  was 
based  a  vision  architecture  developed  in  the  early  1980s  for  the  Pis 
VIDEOPLACE  system.  With  the  four  cameras  currently  in  hand,  it  would  be 
possible  to  cover  a  12’  by  12’  area.  The  primary  disadvantage  of  the 
overhead  approach  is  that  a  high  (16’)  ceiling  is  required.  Side  looking 
cameras  do  not  increase  the  performance  and  in  fact  add  complexity  to  the 
system:  however,  they  fit  more  gracefully  into  the  dimensions  of  most 
laboratories. 

Finally,  the  spatial  resolution  of  both  the  overhead  and  side-looking- 
camera  approaches  can  be  greatly  increased  by  using  the  2Kx2K  sensor 
from  the  same  manufacturer  at  a  small  speed  penalty.  Unfortunately,  by 
the  time  that  we  had  tested  the  512x512  camera  sufficiently  to  be 
satisfied  that  it  would  work  for  us  and  tried  to  purchase  the  high 
resolution  device,  the  manufacturer  had  withdrawn  it  from  the  market 
citing  problems  with  its  performance.  They  promise  that  they  will 
reintroduce  it  but  do  not  have  a  date  certain  at  the  moment. 

V.D.3.  Advantages  of  Random  Access  Cameras 

There  are  a  number  of  advantages  of  the  random  access  camera: 

1.  it  is  a  home  game  for  the  PI  in  terms  of  the  techniques  needed  to 
implement  it 

2.  it  is  compatible  with  the  original  overhead  downward-looking 
camera  approach  and  could  be  fit  into  it  and  the  original  software 
employed. 

3.  its  needs  for  selectable  LED  targets  are  common  to  all  approaches 
now  being  considered 

4.  its  algorithms  for  computing  location  and  orientation  are  also 
common  to  all  approaches 

5.  it  will  give  us  a  short  distance  wireless  tracking  capability  very 
quickly. 

6.  it  can  use  off-the-shelf  lenses. 

V.E.  Smart  Targets 


41 


In  addition,  to  the  design  alternatives  mentioned  above,  there  are  two 
additional  ideas  that  can  be  applied  to  any  of  those  approaches.  One  is 
based  on  the  concern  that  corner-mounted  sensors  viewing  LEDs  from  the 
side  may  suffer  from  blind  spots  when  the  person  is  in  a  corner  and  too 
close  to  a  sensor  for  it  to  see  the  target.  Similarly,  we  have  been 
concerned  that  as  the  person  tilted  his  head  too  far  the  sensors’  view  of 
the  LEDs  would  not  be  optimal.  (Just  as  the  optics  have  a  limited  field  of 
view,  so  do  the  LEDs  and  even  though  we  have  the  widest  field-of-view 
LEDs  procurable,  there  is  less  light  emitted  at  oblique  angles.)  For  any 
configuration  of  LEDs,  there  is  always  a  position  and  head  orientation  that 
will  make  some  of  the  LEDs  hard  to  view  from  fixed  sensors. 

One  solution  to  this  problem  may  actually  increase  the  special  accuracy  of 
the  system  rather  than  simply  preserving  it.  The  idea  is  to  have  redundant 
targets  mounted  around  the  participant’s  head.  At  each  moment,  a  virtual 
target  containing  enough  points  to  define  a  plane  is  created  from  a  subset 
of  these  LEDs  which  are  optimally  positioned  for  viewing  by  the  sensors. 
If  selectable  target  combinations  can  be  made  to  work,  then  we  can 
configure  the  targets  to  be  visible  to  the  nearest  sensor  pair.  If  this  can 
be  done  at  operational  speeds,  it  is  theoretically  possible  to  increase  the 
resolution  by  about  50%.  (In  the  original  approach,  when  the  person  was  in 
one  corner  of  the  space,  one  of  the  sensors  viewing  him  would  be  in  the 
opposite  corner.  By  selecting  a  combination  of  LEDs  that  are  visible  to  the 
two  of  the  other  cameras  that  are  closer,  the  resolution  is  improved.) 

V.F.  Steerable  Sensors. 

Another  way  of  reducing  the  cost  of  the  optics  (or  of  increasing  the 
resolution)  for  all  approaches  is  a  bit  more  elaborate.  The  high  cost  of  the 
line  scan  optics  was  largely  due  to  the  desire  for  a  wide  field  of  view 
(90°)  which  is  necessitated  by  the  need  for  four  detector  systems 
mounted  in  the  corners  of  a  room  to  provide  complete  coverage  of  it.  If  a 
narrower  field  of  view  could  be  made  to  work,  the  optics  would  become 
much  more  pedestrian  and  the  costs  consequently  lower. 

As  noted  above,  it  may  often  be  desirable  to  select  only  two  or  three  of 
the  sensors  for  viewing.  At  such  times,  the  other  sensors  could  be 
mechanically  steered  to  aim  more  directly  at  the  participant.  Since  the 
participant  does  not  move  very  quickly  about  the  physical  space,  1/10  or 
even  2/10  of  a  second  could  be  available  for  such  aiming.  The  benefit 
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would  be  two  fold.  First,  a  cheaper  optical  assembly  with  a  narrower  field 
of  view  (say  30°)  could  be  used.  Second,  the  full  resolution  of  the  line 
scan  sensor  could  be  applied  to  that  narrower  field  of  view  tripling  the 
resolution.  The  horizontal  resolution  would  be  increased  because  the  6000 
pixels  of  the  sensor  would  be  spread  over  30°  rather  than  90°.  The  vertical 
resolution  would  be  increased  because  spreading  the  6000  pixels 
resolution  over  a  90°  vertical  field  of  view — in  the  symmetric  as  opposed 
to  abandoned  anamorphic  design — meant  that  most  of  those  pixels  were 
looking  at  points  in  the  environment  that  were  either  above  or  below  the 
elevation  of  the  participant’s  head.  (A  30°  vertical  field  of  view  was  all 
that  was  deemed  useful  with  the  anamorphic  design.)  Alternatively,  the 
same  resolution  could  be  achieved  with  smaller  sensors  which  require 
smaller  much  cheaper  lenses. 

This  design  approach  raises  several  concerns.  One  is  the  additional 
complexity  of  mechanical  scanning.  A  second  is  the  constraints  that  may 
be  placed  on  the  speed  of  the  participant’s  movements.  (A  participant’s 
movements  around  the  environment  would  have  to  be  simulated  to 
determine  how  severe  those  constraints  would  have  to  be.)  A  third  is  the 
need  for  very  accurate  measurement  of  the  angle  of  the  steering  mirror. 
While  this  approach  is  considered  interesting,  no  action  is  contemplated 
on  its  implementation  at  the  moment. 

V.G.  Current  Tracking  Plan 

As  the  random  access  camera  approach  is  proceeding  we  will  keep  beating 
on  the  optics  for  the  line  scan  approach  which  we  have  now  reduced  to  $8K 
per  unit  for  four  first  stage  optics  systems  and  $3K  each  for  8  copies  of 
the  second  stage  optics.  This  is  within  our  resources;  however,  we  would 
rather  not  invest  so  much  money  in  the  optics  because  these  costs  do  not 
come  down  the  way  electronics  costs  do. 

After  the  comparative  costs  are  worked  out  and  as  we  get  experience  with 
the  short-range  random-access  camera  system,  we  will  decide  whether  to 
build  the  line  scan  optics  or  whether  to  simply  move  to  the  high- 
resolution  random  access  camera  system.  If  the  3M  retro  reflector  does 
not  pan  out  and  the  high-resolution  CMOS  camera  is  not  put  back  on  the 
market,  we  will  use  the  four  low-resolution  cameras  mounted  overhead  to 
create  a  12’  square  space.  In  the  meantime,  the  development  of  the  low- 
resolution  random-access  camera  system  proceeds  apace. 
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VI.  Odor  Studies 

Initially,  psychophysical  studies  examining  the  effect  of  odors  on 
performance  of  surgery- related  tasks  formed  a  significant  part  of  the 
project.  Two  major  studies  were  done  in  the  first  year.  One  tested  the 
effects  of  odors  on  spatial  reasoning  and  the  other  the  effects  of  odors  on 
manual  dexterity.  Both  were  definitive  studies  with  90  subjects  apiece. 
However,  while  the  two  issues  seemed  closely  related,  the  two  studies 
had  completely  opposite  results.  Spatial  reasoning  was  found  to  be 
positively  affected  and  manual  dexterity  not  affected  at  all.  Thus,  the 
results  were  equivocal  and  the  very  large  number  of  subjects  required  to 
swamp  out  individual  differences  made  them  expensive.  Since  odor 
acquisition  had  turned  out  to  be  much  more  difficult  that  expected,  we 
decided  to  reallocate  more  resources  in  this  direction.  When  the 
technology  issues  were  all  resolved,  we  would  then  reconsider  how  much 
could  be  done  in  the  way  of  studies. 

While  the  odor  acquisition  continues  to  be  a  continuing  agony,  the  tracking 
and  advanced  delivery  systems  are  proceeding  with  some  confidence. 
Therefore,  we  have  been  looking  at  a  small  series  of  studies  that  could  be 
done  incrementally. 

One  area  of  interest  is  the  use  of  odors  as  alarms.  There  is  only  one 
existing  example  of  an  odor  being  used  for  this  purpose  and  that  is  the  use 
of  mercaptans  to  signal  the  presence  of  natural  gas  in  the  home.  Even  this 
example  is  in  jeopardy,  because  elderly  people  have  been  found  to  lose 
their  sensitivity  to  this  odor.  However,  there  are  other  situations  where 
odors  signal  emergency  conditions.  The  most  obvious  of  these  is  the  smell 
of  smoke  indicating  a  fire  or  more  specifically  the  smell  of  burning 
insulation  to  signal  a  short  circuit  in  an  electrical  system. 

Odors  could  certainly  be  used  to  signal  such  unique  emergency  conditions. 
In  addition,  they  could  be  used  in  more  subtle  ways.  For  instance,  odors 
could  signal  the  increased  likelihood  of  an  event  or  the  presence  if  not  the 
immediacy  of  a  threat.  They  could  signal  the  state  of  a  system  and 
activate  a  set  of  context-specific  expectations.  They  could  display  the 
vital  signs  of  a  patient  to  a  surgeon  during  surgery. 

While  these  applications  are  probably  feasible,  there  are  a  number  of 
caveats  and  subtleties  involved: 
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1.  odors  are  hard  to  evacuate  from  a  space  and  so  could  not  be  used  for 
commonly  occurring  events  if  they  are  to  be  displayed  to  a  room  full  of 
people.  However,  if  a  person  is  wearing  a  mask  or  even  a  odor  presentation 
device  in  front  of  his  nose,  persistence  is  not  as  much  of  a  problem.  The 
very  small  volumes  of  odorants  used  in  these  cases  and  the  means  of 
evacuation  or  environmental  dilution  assure  that  the  odor  will  be  gone 
from  in  front  of  the  person’s  nose. 

2.  people  adapt  to  odors  quickly  and  then  lose  their  ability  to  smell 
them.  This  feature  means  that  odors  can  signal  transitions  but  are  not  as 
good  at  displaying  a  sustained  context. 

3.  people  differ  wildly  in  their  sensitivity  to  odors  and  in  their 
tolerance  of  them.  Indeed,  individuals  may  literally  have  had  a  bad 
experience  with  a  particular  odor  when  they  were  children  and  can  never 
forget  that  association. 

Assuming  that  odors  can  used  as  alarms  and  as  cues  in  practical 
applications  including  surgery,  what  kinds  of  research  questions  need  to 
be  asked?  The  following  candidates  have  been  identified  so  far: 

1.  what  kinds  of  odors  should  be  used?  For  instance,  pleasant  versus 
unpleasant,  stimulating  versus  soothing,  trigeminal  versus  pure  smell. 

2.  can  odors  break  the  concentration  a  person  who  is  involved  in  a 
highly  involving  task? 

3.  how  many  odors  can  be  identified  with  different  conditions? 

4.  how  long  do  people  remember  the  significance  of  odor  associations 
that  they  have  learned? 

We  spent  time  developing  a  small  set  of  studies  with  the  Dr.  Joel  Warm 
and  Dr.  William  Dember  at  the  University  of  Cincinnati.  These  studies  will 
be  implemented  only  if  the  financial  needs  of  the  tracking  and  delivery 
systems  permit. 

VII.  Next  Years  Effort 

The  final  year  of  the  project  will  see  the  completion  of  the  individual 
technologies  and  and  their  integration  into  a  single  virtual  reality  system. 
These  efforts  will  include: 

1.  completion  of  a  preliminary  wireless  tracking  system  based  on  the 
random-access  cameras. 

2.  completion  of  the  final  wireless  tracking  system  using  either  the 
line  scan  sensors  if  the  3M’s  new  retro  reflectors  are  successful  or  the 
high-resolution  random-access  cameras  if  the  latter  become  available. 
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3.  integration  of  the  tracking  and  odor  delivery  into  a  wireless  virtual 
reality  system  which  permits  natural  ambulation.  This  will  involve  the 
upgrading  of  the  current  HMD  which  is  the  most  inexpensive  available. 

4.  integration  of  the  wireless  advanced  delivery  system  and  possible 
reduction  of  its  weight. 

5.  continuing  acquisition  of  odorant  materials  needed  for  various 
scenarios. 

6.  development  of  the  following  virtual  environment  scenarios  for 
displaying  odors; 

A.  an  easy  odor  environment  such  as  an  outdoor  market  or  a  suburban 

home 

B.  a  surgical  simulation 

C.  a  battlefield  mass  casualty  scenario  set  in  the  countryside 

7.  the  final  documentation  of  the  project 
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Introduction 


The  technology  for  odor  generation  begins  with  a  bulk  liquid  followed  by  its 
evaporation  to  produce  molecules  which  are  subsequently  inhaled  in  the  respiratory  tract. 
An  intermediate  step  to  evaporation  often  involves  atomization  of  the  bulk  hquid  to  produce 
droplets,  followed  by  droplet  evaporation.  Reaction  of  the  molecules  with  the  tissues 
involved  creates  the  sense  of  smell.  A  common  example  is  the  perfume  industry  and  the 
squeeze  atomizers  used  to  vaporize  the  liquid  and  produce  a  fragrance.  The  application  of 
interest  here  is  the  incorporation  of  odors  into  the  artificial  reality  (AR)  experience.  The 
goal  is  to  introduce  odors  which  match  the  sense  of  sight  and  sound  experienced  in  the  AR 
environment.  For  that  purpose,  several  technologies  are  under  consideration,  including 
flow  of  the  liquid  through  a  porous  medium  within  which  the  liquid  evaporates,  and 
atomization  of  the  liquid. 

In  this  brief  report,  we  describe  a  method  for  atomization  of  liquids  that  is  both 
simple  and  cheap  to  use.  We  first  give  a  brief  review  of  the  current  technology  for 
atomization,  then  discuss  the  new  approach.  Advantages  and  problems  are  pointed  out. 
Preliminary  results  are  presented  to  demonstrate  the  concept. 

2.  Liquid  Atomization  Technologies 

A  device  which  disintegrates  a  bulk  liquid  into  droplets  is  termed  an  'atomizer'. 
Atomizer  types  can  be  classified  according  to  the  type  of  energy  used  for  atomization. 
These  include  energy  from  the  liquid  itself  which  is  to  be  atomized;  gas  energy;  mechanical 
energy;  electrical  energy;  or  vibrational  energy.  Bayvel  and  Orzechowski  (1993)  and 
Lefebvre  (1989)  provide  extensive  discussions  of  the  various  technologies  for  atomization 
of  liquid. 

3 .  Vibration-induced  Atomization 

A  little  used  technology  for  atomization  is  based  on  passing  the  liquid  over  a 
transducer  which  vibrates  at  frequencies  to  produce  short  wavelengths  that  result  in  break¬ 
up  of  the  liquid  into  droplets.  This  approach  appears  to  be  suitable  for  applications  that 
require  low  spray  velocities. 

The  precise  mechanism  for  atomization  by  this  vibratory  process  is  a  matter  of 
debate.  It  is  generally  thought  (Lefebvre  1989)  to  include  result  in  the  unstable  growth  of 
capillary  waves  on  the  surface  of  the  liquid  into  macroscopic  ripples  that  eject  liquid  and 
thereby  form  droplets.  Though  the  concept  of  atomization  by  this  approach  has  been 
known  for  over  30  years,  it  has  not  been  extensively  studied.  Only  recently  has 
atomization  that  involves  some  form  of  vibration  excitation  of  the  liquid  been  incorporated 
into  a  commercially  available  atomizer  design. 

Analysis  of  free-surface  waves  generated  by  vibration  excitation  was  developed  by 
Peskin  and  Raco  (1963)  in  terms  of  a  Taylor  instability  mechanism.  The  analytical  results 

show  that  the  mean  droplet  size,  D,  is  related  to  the  surface  tension,  a,  liquid  density,  pi, 
and  frequency,  f,  as 

D  » (47ta/(pif2))l/3 
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Thus,  high  frequency  vibrations  produce  small  droplet  diameters.  Mochida  (1978)  studied 
ultrasonic  atomization  of  distilled  water  flowing  in  a  ultrasonically  excited  tube  at  26Hz.  A 
correlation  for  the  'sauter  mean  diameter'  was  developed  as 

DsmD  «  0.158  (a/pi)0-354  ^j0.303f  Ql0139 

where  Ql  is  the  flow  rate  and  jii  the  liquid  viscosity.  Note  that  frequency  does  not  appear 
in  this  correlation  because  it  was  fixed  at  one  value  in  the  experiments.  The  results  of  Land 

and  Mochida  are  similar  in  that  DLand  (1963)‘>^(<y/pl)-^^  while  DMochida  (1978)°=  l(o/pi)-354. 

Vibration  atomization  has  recently  been  proposed  for  drop  formation  in  a  thermal 
control  device  for  microgravity  cooling  applications  (Smith  and  Glezer  (1996)).  In  this 
approach,  a  single  'large'  droplet  is  placed  onto  the  surface  of  a  piezoelectric  crystal  that  is 
vibrated  at  a  frequency  at  which  surface  wave  disturbances  break  up  the  droplet  into  smaller 
droplets.  The  present  study  further  examines  this  concept  experimentally.  In  the  next 
sections,  we  describe  the  experimental  design  and  procedures  and  discuss  preliminary 
results. 

4 .  Description  of  Experiment 

Figure  1  is  a  schematic  of  the  apparatus  of  the  experimental  design.  It  consists  of  a 
single  'large'  droplet  placed  onto  the  surface  of  a  piezoelectric  crystal,  the  drive  electronics 
to  vibrate  the  cryst^  and  cameras  for  recording  the  results.  The  components  are  the 
following:  a  HP  214B  pulse  generator;  an  HP  54600A  oscilloscope;  the  piezoelectric 
crystal  attached  to  an  aluminum  support;  a  halogen  light  source;  and  a  35  mm  motor  driven 
camera  to  record  the  atomization  process.  In  the  experiments  reported  here,  water  droplets 
were  first  deposited  onto  the  crystal  by  hand  using  a  glass  pipette,  then  the  crystal  excited. 
The  droplets  flattened  into  a  'lens'  shape  or  'sessile'  drop  shape  depending  on  the  volume 
of  water.  For  the  results  reported  here,  the  wetted  diameter  before  excitation  was  from 
4nim  to  6mm.  Some  limited  results  were  attempted  with  an  organic  liquid,  heptane,  but 
were  unsuccessful  because  heptane  rapidly  spread  into  a  thin  film  over  the  surface  of  the 
crystal  due  to  the  comparatively  low  surface  tension  of  heptane  relative  to  water:  about  30 
dyne/cm  to  over  70  dyne/cm). 

Figure  2  shows  a  top  view  of  the  aluminum  housing  used  to  secure  the  piezoelectric 
crystal.  'ITte  crystal  is  attached  to  the  plate  with  10-24  threaded  bolts,  and  flat  washers  at 
four  comers.  This  arrangement  secures  the  edges  and  allows  the  active  area  to  oscillate 
freely.  The  crystal  has  one  lead  soldered  to  the  back  and  another  wire  is  attached  to  one  of 
the  fastening  screws  to  complete  the  circuit.  Excitation  of  the  crystal  was  via  a  pulse 
generator  (HP  214B).  Input  signals  were  observed  on  a  digital  recoring  oscilloscope  (HP 
54600A).  In  all  cases,  a  square  wave  input  was  used.  The  frequency,  voltage,  and  pulse 
width  were  varied.  The  limits  on  these  parameters  was  controlled  by  the  pulse  generator 
until  a  'overload'  condition  was  observed. 

Figure  3  is  a  detailed  schematic  of  the  piezoelectric  crystal.  Dimensions  are 
indicated  in  the  figure.  The  back  side  of  the  crystal  is  brass.  It  was  not  polished  for  the 
experiments  reported  here.  The  crystal  was  oriented  horizontally  with  the  back  side  facing 
upward,  and  test  droplets  were  gently  placed  on  it. 

The  photographic  arrangement  used  to  capture  the  sequence  of  events  consists  of  a 
55  mm/2.8  Nikkor  lens  connected  to  a  PB-6  bellows,  Nikon  F-3  body,  and  a  MD-4  motor 
driver.  The  droplet  is  illuminated  by  direct  back  lighting  using  a  halogen  bulb.  This  set-up 
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created  a  direct  line  of  sight  from  to  the  camera  through  the  droplet.  Film  used  in  these 
preliminary  experiments  was  Kodak  color  print  film. 

5 .  Experimental  Procedure 

The  experimental  procedure  involved  first  depositing  the  test  (water)  droplet  onto 
crystal  and  then  gradually  increasing  the  system  variables  until  an  explosive  atomization 
process  occurred.  The  piezoelectric  crystal  produces  a  ringing  noise  which  corresponds  to 
the  rapid  oscillations  of  the  active  area,  growing  louder  with  increasing  voltage  and  higher 
in  pitch  with  increasing  frequency;  this  gave  us  a  basis  to  determine  our  starting  ranges. 
The  lowest  voltage  amplitude  examined  was  about  20  millivolts  and  the  maximum  was 
33.44  volts  at  which  the  pulse  generator  indicated  an  'overload'  condition. 

To  identify  the  combination  of  variables  in  which  droplet  atomization  occurred,  we 
systematically  varied  the  variables  as  follows.  First,  the  voltage  and  pulse  width  was  fixed 
and  then  the  frequency  was  varied  until  either  the  droplet  exploded  or  an  'overload' 
condition  occurred.  Then,  the  voltage  was  increased  to  a  new  value  while  holding  the 
pulse  width  constant  and  the  frequency  again  increased  until  atomization  or  'overload'. 
After  that  limit  was  explored,  the  pulse  widtfi  was  changed  and  the  process  repeated. 

The  frequency  range  was  limited  by  the  pulse  generator  characteristics;  the  upper 
limit  was  found  to  be  about  3300Hz.  The  optimal  range  was  1500  -  2000  Hz.  The  pulse 
width  ranged  from  0.1  milli-seconds  to  0.25  rmlli-seconds. 

To  capture  the  time  sequence  of  the  droplet  atomization  we  kept  the  voltage  and 
pulse  width  constant,  and  increased  frequency  starting  at  1000  Hz  until  atomization 
occurred.  The  motor  driver  on  the  camera  was  set  at  5  fi*ames  per  second. 

6.  Results 

A  sequence  of  photographs  is  shown  in  figures  4  to  6.  A  color  print  film  was  used 
which  is  the  reason  for  the  pink  'tint'  to  the  images  which  are  largely  devoid  of  any  color 
(future  studies  would  use  black  and  white  film).  The  magnification  of  the  images  are 
evident  for  the  lens  arrangement  used.  An  approximate  scale  is  shown  on  the  figures.  The 
same  nominal  conditions  were  used  for  the  three  sequences  shown. 

Figs.  4a  to  6a  show  an  undisturbed  droplet.  Figures  4b  to  6b  show  surface  ripples 
prior  to  the  shattering  of  the  droplet.  Figs.  4c  to  6c  show  the  dynamics  of  the  droplet 
disintegration  process.  The  speed  of  ejection  is  suggested  by  the  blurring  of  some  of  the 
microc^oplets  produced.  Furthermore,  some  of  the  atomized  droplets  are  in  focus  and 
appear  to  be  more  than  one  order  of  magnitude  smaller  than  the  wetted  diameter  of  the 
original  lenticular  droplet  (figs.  4a  to  6a). 

Figures  7  to  9  show  the  frequency/voltage/pulse  width  conditions  at  which 
atomization  occurred.  Generally,  for  frequencies  below  1400  Hz,  no  atomization  was 
observed.  Similarly,  above  1700  Hz,  an  'overload'  condition  tended  to  be  reached.  It 
should  be  emphasized  that  these  combinations  of  variables  are  likely  to  be  strongly 
dependent  on  the  physical  dimensions  of  the  crystal  used.  Thinner  crystals  can,  for 
example,  produce  atomization  at  lower  frequencies.  Furthermore,  while  an  intense 
'ringing'  was  observed  during  excitation  of  the  crystal,  this  problem  could  be  avoided  by 
using  ultrasonic  frequencies.  The  voltage  and  pulse  widths  needed  to  atomize  the  droplet 
using  ultrasonic  frequencies  will  be  different  than  reported  here. 

7 .  Future  Work 
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•Future  experiments  should  explore  a  higher  frequency  domain  to  avoid  the  audible 
sound  that  can  make  this  technology  prohibitive  for  incorporation  in  the  AR  environment 

•Different  crystal  designs  should  be  examined. 

•The  photographic  process  should  explore  different  lighting  schemes  and  use  black 
and  white  film 

•Finally,  a  system  to  deliver  liquid  to  the  crystal  needs  to  be  automated. 
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Fastening  Bolts 
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Figure  4 


Approximate  scale 


4  mm 


Figure  5 


Approximate  scale 
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Atomization  Response  to  variable  voltages  and  frequencies  at  0.17  milli  second 

width 


Atomization  Response  to  variable  voltages  and  frequencies  at  0.25  milii  second  pulse 

width 


Possibilities  to  Consider  in  Promoting  the  Concept  of 
Aroma  Enhanced  Virtual  Reality 

(The  following  report  is  based  on  the  ideas  discussed  during 
roundtable  meetings  on  the  subject  of  aroma  enhanced  virtual  reality 
held  in  late  Jan.  1998.  Leader  of  the  discussions  was  Dr.  Terry  Acree, 

Professor  of  Biochemistry  at  Cornell  University  and  head  of  the 
Flavor  Labs  at  the  New  York  State  Agricultural  Experiment  Station  in 
Geneva,  New  York.  Contributing  to  the  discussions  were  five 
graduate  students  from  the  Cornell  flavor  labs:  Jane  Friedrich,  Peter 
Ong,  Grace  Feng,  Olivier  Perrin,  and  Kathryn  Deibler.  Also  present 
was  flavor  labs  reasearch  support  specialist  Ed  Lavin  who  compiled 
this  report) 

There  was  a  concensus  from  the  group  that  the  concept  of  adding  the 
dimension  of  odor  to  a  virtual  reality  system  is  both  a  worthwhile 
and  exciting  one.  To  better  consider  the  concept,  the  group  decided  to 
focus  its  ideas  into  two  categories:  promotion  and  execution.  This 
report  will  deal  each  of  these  individually. 

1.  Promoting  the  Idea 

If  the  overall  goal  of  your  group  is  to  research  and  then  demonstrate 
the  impact  that  aroma  could  have  when  integrated  into  a  virtual 
reality  system,  perhaps  the  most  effective  way  to  achieve  this  would 
be  to  produce  a  small  scale  aroma-enhanced  "virtual  reality"  video 
using  odor  delivery  systems  that  are  already  in  use  in  industry.  Our 
group  feels  that  an  aroma  enhanced  video  could  be  made  employing 
scenarios  that  would  include  a  number  of  conunon  everyday  life 
situations.  Inherent  to  these  situations  would  be  moments  where  a 
specific  aroma  would  be  delivered  to  the  viewer  that  would  clearly 
enhance  the  one-dimensional  nature  of  the  scene  on  the  screen.  Such 
a  video  could  demonstrate  the  positive  impact  that  adding  the  extra 
dimension  of  aroma  would  have  on  a  total  virtual  reality  package. 

We  are  not  familiar  with  the  current  state  of  the  art  in  aroma 
delivery  for  these  purposes.  Basic  to  the  premise  of  course  is  that  a 
method  would  have  to  be  devised  to  deliver  and  vacate  the  aromas 
used  and  to  synchronize  them  with  the  on-screen  activity.  It  is 
probable  that  current  options  consist  of  the  old  scratch  and  sniff 
method  in  which  the  viewer  responds  to  an  on-screen  cue  by 
scratching  and  sniffing  a  card  containing  the  various  odors.  Another 
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and  more  attractive  method  would  be  a  system  which  delivers 
airborne  puffs  of  odor  chemicals  directly  to  the  viewer  from  a 
remote  location. 

Aroma  delivery  systems  aside,  our  expertise  is  in  the  realm  of  food 
science.  The  vast  majority  of  known  aroma  producing  compounds 
are  associated  with  foods,  therefore  our  suggestion  is  to  make  a  video 
which  displays  the  enhancement  aroma  lends  to  food.  To  give  you  an 
idea  of  what  we  mean  we  have  scripted  a  video  sequence  which  will 
demonstrate  the  concept  of  aroma  enhanced  virtual  reahty.  The 
compound  name  and  the  approximate  amount  that  would  be  needed 
to  produce  the  desired  effect  is  noted.  This  script  is  a  rough  sketch 
which  can  be  used  to  demonstrate  aroma  enhanced  virtual  reality. 

Our  script  begins  with  a  medical  professional  on  his  day  off  doing  an 
everyday  task  and  ends  with  his  re-entering  the  professional  realm 
of  medicine.  It  was  thought  this  might  be  an  effective  way  to  bring 
in  the  medical  angle  which  is  ultimately  the  driving  force  behind  this 
research. 

Scene  1  (SI)-  Doctor  in  home,  says  good-bye  to  wife  "going  down  to 
the  market" ,  exits  front  door.  Says  good-  bye  to  son  who  is  mowing 
grass.  >  Aroma  1  (Al)  hexanal  (smell  of  fresh  mown  grass— 
25ppm) 

S2-  Physician  goes  to  car  and  gets  in  >  (A2)  vinylchloride  (new  car 
smell) 

53  -  Pulls  out  of  driveway,  then  pops  a  piece  of  gum  in  his  mouth  > 
(A3)  carvone  (peppermint  odor  —  50ppm) 

54  —  Arrives  at  open-air  market  in  a  park  like  setting. 

S5—  Sees  popcorn  stand  near  market  and  stops  to  buy  popcorn  > 

(A4)  2-acety-l-pyrroline  (popcorn  aroma—  50ppb) 

56  -  Stops  at  cheese  stand  >  (A5)  butyric  acid  (cheesy—  5ppm) 

57  -  Walks  by  cotton  candy  vendor  >  (A6)  furaneol  (burnt-sugar 
smell  of  cotton  candy  —  25ppm) 

58  —  Walks  by  peanut  vendor  >  (A7)  2-acety  pyrazine  (roasted 
peanut-lppm) 


Appendix  8 -61 


S9  -  Stops  at  fruit  vendor,  buys  apple,  takes  a  bite  >  (A8)  ethyl 
butyrate  &  E-2-hexenal  (apple  and  apple  peel—  lOppm) 

SIO  ~  Checks  out  orange  stand,  buys  a  dozen  oranges  >  (A9)  orange 
oil  (1%  by  vol.) 

SI  1  —  Stops  by  bell  pepper  stand  >  (A  10)  3-methoxy-2- 
isobutylpyrazine  (green  bell  pepper  aroma  —  lOppb) 

512  "  Stops  by  fish  vendor-sniffs  fish  to  test  for  freshness.  >  (A1 1) 
trimethyl  amine  (unfresh  fish  aroma  —  lOppb) 

513  —  Stops  at  vendor  and  buys  a  cup  of  concord  grape  juice  >  (A12) 
methyl  anthranilate  &furaneol  mixture  (concord  grape  50ppm) 

514  —  Tries  a  sample  slice  of  cucumber  >  (A13)-  E-2-nonenal  (green 
cucumber  like  —  Ippm) 

515  -Approaches  mushroom  vendor  >  (A14)  l-octen-3-one  (fresh 
mushroom  like  —  Ippm) 

SI 6-  Stops  and  eats  a  banana  >  (A15)  isoamyl  acetate  (banana 
aroma-lOppm) 

S17  —Buys  chocolates  for  his  kids  >  (A16)  vanillin  (lOOppm) 

SI 8-  Buys  roses  from  vendor  for  wife  >  (A17)  2-phenylethyl  alcohol 
( rose -like  aroma  -  SOppm) 

5 19  —  There  is  a  sudden  commotion.  Sirens  are  heard  and  an 
ambulance  pulls  up  in  a  park  adjoining  the  market.  Doctor  moves  to 
help  and  comes  upon  a  scene  with  a  man  down,  apparently 
unconscious  surrounded  by  onlookers  and  EMT's.  Doctor  identifies 
himself  and  begins  to  work  on  the  man. 

520  —  Doctor  begins  to  check  for  vital  signs.  He  rips  open  the 
patient's  shirt  and  puts  his  face  close  to  body  of  patient  to  check  for 
breathing  and  pulse.  >  (A18)  hexanoic  acid  (body  odor  aroma— 
SOOppm) 
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521  —  Doctor  loosens  victim's  belt  to  assist  breathing.  Further 
examination  shows  patient  has  urinated  >  (A19)  phenylacetic 
acid(urine  smell  -  lOOppm) 

522  —  Patient  remains  unconscious  and  appears  to  be  going  into 
cardiac  arrest.  Doctor  decides  to  begin  CPR,  as  is  often  the  case  the 
patient  aspirates  during  the  procedure  >  (A20)  butyric  acid  (vomit 
aroma-  500ppm) 

523  ~  Patient  is  loaded  onto  a  gurney  and  doors  of  the  ambulance 
open  and  patient  loaded  on  with  doctor  climbing  on  board  >  (A21) 
heliotropin  (odor  of  cleaning  solutions  associated  with  sterile 
environments  like  hospitals  etc.) 

524  —  Doors  close  on  the  ambulance.  Text  on  screen;  To  be 
Continued. 

End  of  Video 


Discussion 

If  the  video  and  aroma  concepts  are  produced  correctly,  the  video 
will  demonstrate  the  positive  effect  of  aroma  when  coupled  with 
video.  But  to  drive  home  the  point  that  not  much  is  known  about  the 
olfaction  of  medical/surgical  environs,  the  video  ends  with  the  words 
"To  Be  Continued"  rather  than  "The  End".  We  feel  this  will  drive 
home  the  point  that  the  idea  of  aroma-sustained  virtual  reality  is  a 
good  and  exciting  concept  but  one  that  needs  a  lot  of  development  to 
make  a  reality.  Our  group  feels  that  this  type  of  ending  could  be  an 
effective  lead-in  to  discussions  including  prospective  project 
supporters,  developers,  and  of  course,  funders. 


n.  Executing  the  Concept:  Is  it  Feasible? 

It  is  important  to  note  that  the  aromas  that  would  be  delivered  in 
the  aforementioned  video  were  the  result  of  many  years  of  research 
by  food  scientists  throughout  the  world.  Their  efforts  to  isolate  the 
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trace  active  aroma  constituents  in  the  complex  matrices  of  natural 
foods  took  generous  amounts  of  time,  effort  and  of  course,  money.  In 
addition,  the  chemicals  listed  were  only  the  major  character 
compound  for  each  of  these  foods.  There  are  many  lesser  compounds 
that  add  the  complexity  and  a  three  dimensional  nature  to  real  food 
aromas.  Considering  that,  the  challenge  of  capturing,  isolating,  and 
identifying  the  multitude  of  smells  associated  with  surgical/medical 
procedures  is  better  understood.  There  are  many  questions  to 
consider.  For  example: 

How  do  we  isolate,  capture ,  concentrate,  and  identify  the  important 
components  in  an  odor  event  of  interest? 

Odor  compounds  are  usually  present  in  quite  minuscule  amounts 
(ppb  to  low  ppm  levels)  when  compared  to  other  naturally  (or 
unnaturally)  occurring  compounds  in  any  given  environment. 

Samples  may  contain  many  scores  of  compounds  of  which  only  a 
handful  have  any  odor  at  the  levels  they  are  present.  Allowing  that 
we  are  able  to  gather  enough  sample  from  the  airspace  to  do  this 
analysis  (of  course  there  are  also  issues  of  accessibility  to  surgical 
settings,  bureaucratic  and  ethical  red  tape,  insurance  issues  etc.),  it 
will  be  necessary  to  do  extensive  bioassay  work  to  determine  the 
most  important  characterizing  candidate  compounds.  Capturing 
enough  of  these  airborne  volatiles  to  do  an  effective  analysis  will  be 
a  challenge  although  recent  introduction  of  Solid  Phase  Micro 
Extraction  (SPME)  technology  which  provides  a  portable  and  cost 
effective  method  of  sampling  for  airborne  volatiles.  Once  samples 
are  collected,  bioassay  work  will  be  necessary  to  begin  the  process 
of  identifying  the  major  character  impact  compound(s)  in  a  given 
sample.  This  bioassay  will  likely  entail  use  of  Gas 
Chromatography/Olfactometry  (GCO),  a  system  that  employs 
feedback  from  human  "sniffers"  and  sorts  through  a  mixture  of  odor 
compounds  in  a  sample  and  rates  their  odor  potencies.  Identification 
of  unknowns  and  verification  of  known  odor  compounds  will  depend 
on  good  GC/MS  (Mass  Spectrometry)  and  high  quality  mass  spectral 
libraries  for  data  interpretation.  In  addition,  it  may  be  necessary  to 
employ  Nuclear  Magnetic  Resonance  (NMR)  to  verify  identifications 
done  via  GC/MS  . 

Another  possible  source  of  information  will  come  from  literature 
searches  carried  out  using  the  multiplicity  of  medical  research 
sources  available.  It  may  be  possible  to  gather  some  useful 
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information  on  research  already  done  on  certain  organs  which  may 
act  as  guides  as  we  try  to  identify  odor  active  agents. 


Once  identified,  how  do  we  find  a  source  of  the  compound  or  how  to 
synthesize  it? 

It  is  hkely  that  compounds  we  will  identify  are  not  commercially 
available  for  purchase.  Those  compounds  not  available  will  need  to 
be  synthesized.  This  can  be  a  tricky  and/or  expensive  proposition. 
One  resource  is  to  look  for  a  commercially  available  analogue  or 
closely  related  species  of  chemical  and  determine  its  acceptability  as 
a  substitute.  If  there  is  not  an  acceptable  alternative,  it  will  be 
necessary  to  contact  a  commercial  synthetic  laboratory  and 
investigate  the  feasibility  of  synthesizing  the  target  compound(s). 


How  do  we  set  the  intensity  levels  of  odors  to  match  those  found  in  a 
real  situation? 

One  of  the  real  challenges  in  re-creating  an  aroma  profile  in  a  virtual 
reahty  type  setting  would  be  the  fact  that  there  is  huge  variation  in 
the  abilities  of  average  people  to  smell.  The  actual  variation  can  be 
measured  in  orders  of  several  magnitudes  where  one  person  will 
have  one  hundred  times  more  sensitivity  to  a  compound  than 
another  person.  The  challenge  will  be  to  deliver  doses  of  odor  that 
are  detectable  to  a  wide  range  of  subjects  but  don't  lose  their 
authentic  character  due  to  over-dose  saturation. 


What  type  of  delivery  system  would  be  appropriate? 

We  are  not  aware  of  any  specific  delivery  systems  in  use  at  this  time 
One  of  our  staff  has  seen  a  system  used  by  Toyota  which  delivers  a 
stream  of  air  through  a  manifold  system  and  delivers  it  to  an  outlet 
on  the  headrest  of  the  chair  to  the  front  of  the  subject.  This  was 
found  to  be  an  effective  method  of  delivery  in  a  room  with  multiple 
stations.  The  issues  here  are  precise  delivery  and  a  system  that 
moves  the  odor  through  without  leaving  tailing  "ghosts"  of  odor 
behind  to  contaminate  the  next  odorant.  Special  care  must  be  taken 
to  ensure  the  use  of  non-odorabsorptive  materials  in  the  hardware  of 
the  system. 
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Will  it  be  possible  to  produce  aroma  that  demonstrates  a  gradient  or 
very  subtle  change  in  odor  from  one  part  of  a  surgery  to  another? 

Such  a  subtle  action  will  be  very  difficult,  although  not  impossible  to 
achieve.  The  best  way  to  achieve  this  would  be  by  changing  delivery 
levels  of  an  aroma.  Effectively  delivering  odors  showing  minor 
differences  will  prove  challenging  due  to  differing  acuity  levels  and 
saturation  and  acclimatization  factors. 


Summary  of  Technical  Issues 


To  summarize,  we  see  the  challenges  to  be  faced  in  producing  aroma 
enhanced  virtual  reality  in  a  surgical/medical  setting  to  be  the 
following: 

Challenge  #1 

Devising  a  system  to  capture  the  odor  events  as  they 
happen  in  the  surgical  environment. 

This  could  mean  using  SPME  devices  in  and  around  surgeries  of 
various  types  and  performing  bioassays  to  identify  the  odor-active 
components.  The  challenge  will  be  to  get  the  right  sample  from  the 
right  place  at  the  right  time  and  to  get  enough  of  the  volatile 
compounds  in  the  short  time  available  to  collect  the  sample  to  do  an 
effective  bioassay.  Currently,  the  SPME  system  is  effective  for 
gathering  a  fairly  wide  range  of  compounds  from  an  airspace. 

However,  it  has  not  to  our  knowledge  been  adapted  for  the  dilution 
analysis  techniques  of  bioassays  like  CharmAnalysis.  Since  the  SPME 
system  is  an  equilibrium-based  collection  method,  it  is  not  especially 
well  suited  for  a  dilution-based  bioassay.  It  appears  that  a  method 
employing  the  split  injection  mode  of  the  gas  chromatograph  injector 
would  have  to  be  developed  to  make  this  technology  compatible  with 
a  dilution-based  bioassay  such  as  CharmAnalysis. 

Challenge  #2 

Identifying  trace  level  odorants  determined  by 
CharmAnalysis  or  some  similar  bioassay  using  GC/MS,  NMR, 
or  some  combination  of  similar  analytical  tools. 
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Most  odorants  are  present  at  trace  levels  therefore  identifying  them 
for  use  in  an  aroma  delivery  system  will  be  labor  intensive.  For 
many  compounds  the  human  nose  is  more  sensitive  than  the 
instruments  we  have  to  measure  them.  The  Charm  bioassay  may  be 
able  to  tell  us  where  to  look  for  answers  on  a  GC/MS  chromatogram 
but  it  does  not  provide  structural  information.  Often  in  areas  where 
odor  is  detected  by  charm,  the  GC/MS  will  detect  nothing  thus 
making  compound  identification  difficult  or  impossible.  The 
challenge  thus  will  be  to  collect  enough  of  the  trace  level  odor-active 
compounds  to  be  identifiable  using  standard  analytical  tools 
available.  The  SPME  technology  is  unproven  but  promising  in  this 
regard. 


Challenge  #3 

Identifying  and  reproducing  subtle  odor  changes  as  they 
occur. 

If  the  task  in  virtual  reality  is  to  reproduce  odors  of  very  different 
types  and  intensities,  the  goal  is  more  achievable.  If  the  goal  is  for 
doing  this  with  very  subtle  changes  that  are  occurring,  the  task  is 
much  more  formidable.  Subtle  changes  in  odor  character  can  be  very 
difficult  to  detect  instrumentally  and  thus  may  be  difficult  to 
reproduce. 


Challenge  #4 

Delivery  of  odor  compounds  to  the  subject. 


The  goal  here  will  be  to  construct  a  system  that  will  deliver  packets 
of  aroma  at  the  correct  intensities  in  an  appropriately  humidified 
airstream  with  no  carryover  from  previously  delivered  odors.  The 
system  will  require  some  type  of  manifold  delivery  design  with 
digital  feedback  etc.  to  accomodate  the  interactive  elements  of 
virtual  reality  where  the  subject  is  allowed  to  make  free  choices  in 
his  movements  and  activities. 


Appendix  Ifi--  67 


ni.  Conclusions 


Based  on  our  discussions,  challenges  1-3  (above)  are  formidable  but 
for  the  most  part  should  be  achievable  with  the  right  amount  of 
research  support.  Challenge  #4  should  not  prove  to  be  an  obstacle 
since  elements  of  this  type  of  odor  delivery  system  are  already  in 
use  at  exhibits  around  the  world.  One  of  those  systems  could  be 
adapted  for  your  purposes. 

Overall,  the  largest  stumbling  block  for  the  surgical/medical 
application  of  aroma  enhanced  virtual  reality  will  probably  be  the 
quahty  of  the  delivered  smells.  The  necessary  level  of  authenticity 
of  the  delivered  odors  will  determine  how  much  effort  needs  to  be 
expended  to  achieve  the  goal  of  an  aroma  enhanced  virtual  surgical/ 
medical  reality  system.  It  may  well  be  that  the  odors  which  you  will 
be  dealing  with  are  intense  and  will  be  more  easily  identifiable  than 
we  currently  imagine.  It  really  becomes  more  of  a  matter  of  scope. 

If  the  goal  is  to  provide  odors  associated  with  antiseptics  and  gross 
bodily  functions  such  as  body  odor,  bowel,  urine  etc.  then  the  current 
knowledge  exists  to  achieve  this  relatively  easily  and  inexpensively. 
However  if  the  aim  is  to  reproduce  other  more  subtle  and  obscure 
odors  associated  with  surgery,  the  playing  field  changes  shape  a  bit. 
As  food  scientists  we  do  not  have  the  requisite  experience  with 
surgical/medical  odors  to  make  a  definitive  statement  as  to  the  true 
feasibihty  or  cost  of  effort  that  will  be  necessary  to  produce 
authentic  surgical  aromas.  However,  based  on  our  experience  with 
identifying  food  flavors  and  aromas  the  challenge  of  identifying 
odors  produced  by  human  organs  during  surgery  may  be  the  biggest 
challenge  confronting  developers  of  an  aroma  enhanced  virtual 
surgery/medical  reality  system. 
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To:  Virtual  Reality  Corp. 

Box  786 

Vernon,  CT  06066 

Simulated  Odors:  Methods  and  Possibilities 

Introduction 

Conceptually,  it  is  attractive  to  think  of  the  simulation 
task  as  comprising  analysis,  reconstitution,  and  delivery  at  an 
appropriate  concentration. 

An  alternative,  perhaps  easier,  approach  would  be  the 
selection  of  a  reference  odorant  to  "stand  for"  the  real  odorant 
and  be  accepted  as  such.  I  will  discuss  the  pros  and  cons,  as 
well  as  the  history,  of  both  approaches. 

This  report  will  also  deal  with  the  task  of  removal  of 
odorants  by  ventilation  or  other  methods. 

I  will  also  discuss  some  of  the  odorants  of  specific 
interest  to  you. 

Analysis  and  reconstitution:  single-component*  odorants 

The  task  would  seem  to  be  easiest  for  odorants  such  as 

ammonia,  hydrogen  sulfide,  vanillin,  isovaleric  acid  (for  body 

odor)  ,  and  the  like.  There  are  two  caveats  here:  Is  your 

"single-component"  substance  compromised  by  impurities,  and  does 

it  smell  like  the  "real"  odor  you  are  simulating? 

*I  prefer  the  terms  "single-component"  and  "multi-component"  to 
"simple"  and  "complex,"  which  have  different  chemical  meanings. 
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Impurity  is  a  problem  when  the  contaminants  have  different 
and  stronger  odors  than  the  major  component.  So  if  you  wanted 
the  odorant  to  be  pure  n-octane,  and  it  were  actually 
contaminated  with  the  other  17  octanes  (I've  smelled  them  all), 
it  would  make  little  difference,  because  they  all  have  mild, 
paraffinic  odors.  If  the  substance  of  interest  has  a  very  strong 
odor,  and  the  impurity  does  not,  there  may  be  no  problem,  as 
would  be  the  case,  for  example,  with  a  sulfide  contaminated  with 
a  paraffin.  But  my  experience  with  n-octyl  benzene,  which  I 
selected  as  a  reference  odorant  for  the  "oily"  note  of  diesel 
fuel,  (as  I  discussed  with  you  by  phone)  included  a  batch  that 
had  another  strong  non-oily  odor.  I  traced  this  to  a  different 
synthesis  batch,  but  the  nature  of  the  difference  was  never 
disclosed.  Another,  chemically  more  interesting,  example  was  on 
the  occasion  of  a  NATO-sponsored  Summer  School  on  research 
methods  in  odor  perception,  which  involved  working  with  mixtures 
of  acetophenone  and  eugenol.  A  study  of  the  presumably  pure 
acetophenone  (Turk  and  Turk,  1975)  showed  that  it  was 
contaminated,  and  that  the  impurity  seriously  changed  the  odor 
quality. 

'■  What  should  be  done  to  avoid  such  difficulties?  It  is  easy 
to  demand  "pure"  materials,  but  that  is  not  always  reliable,  and 
extensive  analysis  and  purification  (as  by  zone-refining)  is  not 
generally  feasible.  My  suggestion  is  first,  of  course,  to 
obtain  the  most  reliable  "pure"  sample  available,  then  to 
sequester  an  ample  supply  of  it  as  a  reference  against  which  to 
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match  later  batches.  Ideally,  the  base  supply  should  be  stored 
under  refrigeration,  preferably  in  separate  sealed  ampoules,  but 
do  the  best  you  can. 

Next  question  is,  "Does  your  pure  odorant  smell  like  the 
real  odor  you  are  simulating?”  Sometimes  it  does.  For  example, 
isovaleric  acid  is  said  to  smell  like  human  body  odor,  and 
especially  foot  odor.  In  a  recent  legal  dispute  regarding 
commercial  foot  deodorizing  products,  both  sides  agreed  that 
control  of  isovaleric  acid  odor  was  a  fair  basis  for  comparison. 
And  some  years  ago,  when  my  wife  and  I  were  hiking  alone  in 
Glacier  National  Park,  we  smelled  body  odor.  We  traced  it  to 
some  withered  valerian  flowers  along  the  trail.  (Many  common 
chemical  names  are  of  botanical  origin.) 

Sometimes  it's  close  enough.  For  example,  dimethyl  sulfide 
would  be  a  good  standard  to  represent  the  odor  from  a  pulp  mill, 
even  though  it  is  only  one  of  23  identified  from  such  a  source 
(Himberg  et  al.,  1987),  because  it  is  a  major  component  and  would 
not  be  overwhelmed  by  other  more  highly  odorous  or  very  different 
odor  notes. 

Analysis  and  reconstitution:  multiple-component  odorants 

A  frequent  recommendation  is  to  use  GC-MS  analysis  to 
identify  the  components,  then  to  reconstitute  the  mixture  from  a 
supply  of  pure  odorants.  Obviously,  only  the  odor-relevant 
components  would  be  needed,  so  these  would  have  to  be  identified 
—  possibly  by  sniffing  the  exit  stream  from  the  chromatograph. 
Is  such  an  approach  ever  successful?  From  my  knowledge — 
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rarely,  and  only  with  considerable  effort.  One  successful  effort 
was  by  Roy  Teranishi,  a  chemist  with  the  U.S.  Dep't  of 
Agriculture,  who  was  able,  after  much  effort,  to  reconstitute  the 
flavor  of  green  apple  essence  with  three  odor-relevant 
components.  (I  do  not  have  the  reference,  but  it  could  be 
searched.  He  told  me  that  his  boss  asked  him  why  he  had  to 
analyze  for  all  those  other  non-odorous  components.)  But  when  a 
substance  undergoes  partial  decomposition  or  other  conversion,  as 
by  cooking,  scorching,  smoldering,  burning,  exploding, 
putrefaction,  or  fermentation,  the  product  practically  always  is 
much  more  odorous  and  has  more  components,  often  a  great  many 
more.  These  are  the  mixtures  that  have  more  interesting  odors 
and  are  much  more  difficult,  often  hopelessly  so,  to 
reconstitute.  I  know  of  no  instance  where  such  a  reconstituted 
odor  is  indistinguishable  from  the  original.  The  task  is  further 
complicated  by  the  circumstances  under  which  the  real  odorant  is 
sensed:  (1)  The  substance  may  be  in  more  than  one  state,  such  as 
vapor,  liquid  droplet,  solid  particle,  dissolved  in  the  liquid, 

.  or  adsorbed  on  the  particle.  (2)  The  substance  may  be  unstable  or 
I  otherwise  reactive,  so  that  the  reconstituted  mixture,  even  if 
I  representative,  may  be  difficult  to  preserve. 

Reference  odorants 

It  would  be  convenient  if  you  could  represent  a  real  odor  by 
a  known  substance,  either  a  pure  material  or  a  mixture  of  a  few 
components,  that  was  stable  and  reproducible.  But  what  if  it 
didn't  smell  right?  The  history  of  the  so-called  "Turk  diesel 
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odor  kit”  is  worth  summarizing  in  this  regard. 

The  increasing  awareness  in  the  1960s  of  odors  as  an  air 
pollution  problem  led  the  petroleum  and  diesel  engine  industries 
to  focus  their  attention  on  diesel  exhaust  odors.  It  was 
recognized  that  a  method  of  measuring  these  odors  had  to  be 
devised  before  control  techniques  could  be  evaluated.  Early 
attempts  to  use  category  scales  of  perceived  intensity  (weak, 
moderate,  strong,  etc)  were  unsatisfactory:  they  did  not  yield 
reproducible  results  and  did  not  address  the  different 
"qualities”  of  the  exhaust  odors.  I  suggested  that  an  overall 
reference  standard  be  developed  that  would  serve  to  anchor 
different  intensities  of  diesel  exhaust  odor,  and  that  the 
various  "qualities”  of  the  odor  be  referenced  by  individual 
standards  (Turk,  1970) .  The  number  of  such  quality  standards  was 
limited  by  the  sensory  capacity  of  judges  to  distinguish  among 
them.  Perfumers,  despite  their  reputations,  cannot  factor  out 
hundreds  of  odor  notes  in  a  mixture  —  they  usually  work  with 
multiple  paired  comparisons.  I  concluded,  after  tests  and 
interviews,  that  a  workable  number  of  different  sensory  odor 
qualities  was  four.  The  next  task  was  to  define  and  make  up  the 
separate  quality  standards  and  the  overall  "diesel  odor” 
standard.  We  smelled  diesel  exhaust  under  different  conditions 
of  engine  operation  (idling,  cruising,  acceleration,  and 
deceleration)  and  prepared  concentrates  of  them.  We  also 
identified  the  most  commonly  described  qualities  of  diesel  odor 
as  burnt/ smoky,  oily,  pungent/acid,  and  aldehydic/aromatic.  The 
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only  single-component  quality  selected  was  n-octyl  benzene  for 
the  oily  standard.  The  overall  diesel  odor  standard  consisted  of 
a  mixture  of  three  of  the  quality  standards.  Each  of  the  quality 
reference  standards  was  compounded  in  four  different  dilutions, 
and  the  overall  diesel  standard  in  12  dilutions,  making  a  "kit" 
of  16  +  12,  or  28  standards  in  the  kit. 

How  well  did  the  kits  serve?  They  were  used  in  extensive 

studies  by  Southwest  Research  Institute  and  by  various 

manufacturers  of  diesel  engines  and  fuels.  No  one  thought  the 

diesel  standard  smelled  just  like  diesel  exhaust,  but  none  of  the 

j!  test  groups  had  any  difficulty  in  relating  the  standard  to  the 

j  real  thing.  The  quality  standards  were  also  very  useful  in 

distinguishing  between  odors  from  different  fuels  and  different 

modes  of  operation.  Shortly  after  this  work,  I  was  asked  by  the 

City  of  New  York  to  evaluate  the  effect  on  the  odor  from  its 

diesel  buses  of  using  a  catalytic  converter  offered  by  a 

O'f- 

supplier.  Our  results,  obtained  with  the  aid.^^my  odor  panel  and 
the  diesel  kit,  showed  clearly  that  the  converter  would  reduce 
the  overall  diesel  odor  intensity  and  the  burnt  quality  of  its 
odor,  but  would  increase  the  perceived  pungency.  I  recommended 
it  not  be  used,  and  the  City  rejected  it. 

What  do  these  findings  imply  for  your  "virtual  reality" 
objectives?  In  my  opinion,  subjects  can  comfortably  associate 
an  ^  artificially  simulated  standard  with  a  real  odor  if  the 
I  simulation  is  not  too  far-fetched  and  is  not  otherwise  strongly 
reminiscent  of  a  different  familiar  odor.  The  latter  point  is 
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important.  For  the  "oily”  quality  of  diesel  exhaust  I  could 
have  selected  linseed  oil,  or  olive  oil,  for  example.  But  these 
are  too  familiar,  and  it  would  be  difficult  to  reassociate  them 
with  another  odor.  N-octyl  benzene  is  not  familiar,  so  no 
transfer  had  to  be  made,  and  there  was  no  problem  with  the  new 
association.  Similarly,  I  would  not  use  burnt  meat  for  the 
burnt  diesel  odor  quality,  etc. 

The  ease  of  sensory  association  of  different  odor  qualities 
for  the  purpose  of  testing  was  taken  a  step  further  with  the 
development  of  the  butanol  odor  standard  (Moskowitz  et  al., 
1974) .  This  standard  is  now  the  basis  for  an  A.S.T.M.  method  and 
is  widely  used.  Judges  have  no  difficulty  in  rating  the 
perceived  intensity  of  an  odor  with  the  intensity  of  a  butanol- 
air  mixture.  Dravnieks'  (1985)  compendium  of  odor  quality 
profiles  offers  a  tempting  approach  to  selecting  "virtual"  odor 
4  standards.  I  think  you  should  have  a  copy,  and  it  may  be  a 
moderately  helpful  guide,  but  its  value  is  limited  and  much  of 
the  data  will  be  confusing. 

Mv  recommendation  on  odor  simulation 

It  would  be  very  helpful  if  reference  odors _ that  do  not 

duplicate  the  real  ones  would  work  for  you.  They  do  work  for 
odor  measurement,  and  for  gas  odorization  (Cain  and  Turk,  1985) , 
but  no  one  has  tested  them  for  your  application.  You  should  do 
such  testing.  If  they  are  accepted,  you  will  be  blessed.  You 
would  then  be  able  to  make  and  store  a  supply  from  which  aliquots 
could  be  selected  as  needed. 
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Dissemination  and  removal 


The  dissemination  options  include  scratch-and-snif  f , 
permeation  tubes  (for  a  steady  dose) ,  and  diffusion  through  a 
capillary.  I  am  not  familiar  with  the  mechanics  of  your  systems, 
but  these  or  other  methods  could  be  tested. 

Removal  may  be  by  ventilation  or  activated  carbon 
adsorption.  The  latter  method  is  very  rapid,  once  the  vapors 
reach  the  carbon,  but  the  changes  of  vapor  concentration  with 
time  in  a  space  are  subject  to  the  constraints  of  chamber 
volume,  mixing  efficiency,  and  air  flow  rate  (Turk,  1963  —  the 
photo  was  old  even  then  —  and  1968) . 

Your  statement  of  work 
Diesel  and  gasoline 

Diesel  fuel  can  be  readily  simulated  as  per  the  reprint 
enclosed,  with  the  caveats  regarding  reproducibility  as  discussed 
above.  Toxicity  is  very  low  for  intermittent  and  occasional 
exposure.  Gasoline,  now  often  referred  to  as  BTEX  (for  benzene, 
toluene,  ethylbenzene,  and  xylene)  contains  these  components  as 
well  as  the  mostly  C7  and  €3  paraffins,  olefins,  and  alicyclics, 
and  MTBE  (methyl-t-butylether) .  Most  people  don't  have  much 
occasion  to  smell  gasoline  now,  with  the  new  recycling  hoses  at 
filling  stations,  so  a  reasonable  hydrocarbon  simulation  at  low 
toxicity  should  not  be  difficult  to  do.  It  should  be  easy  to 
make  100  mL  or  more  of  each.  Particulate  matter  is  irrelevant 
here,  because  odor  exposure  is  only  to  vapors. 
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Lubricating  oil 

Not  difficult  to  simulate,  but  the  odor  is  mild  unless  the 
oil  is  hot.  Simulating  hot  oil  at  ambient  temperature  could  be 
tricky,  but  I  believe  it  could  be  done. 

Explosives 

Mild  odor  from  TNT  mostly  from  undernitrated  toluene  and 
some  oxidation  by  the  nitric  acid,  which  does  more  than  nitrate 
the  toluene.  The  "red  water”  effluent  from  TNT  manufacture 
contains  some  of  these  by-products.  Also  some  sulfur  notes  from 
the  reduced  H2SO4  in  the  mixed  acid.  RDX  and  HMX  are  white 
crystals,  practically  odorless.  Nitroglycerin  and  particularly 
EGDN  (ethylene  glycol  dinitrate) ,  both  in  dynamite,  can  be 
smelled  by  dogs,  but  not  much  by  people.  Ammonium  nitrate  is  a 
crystalline  salt  —  no  odor. 

It*s  different  when  they  explode,  of  course.  The  resulting 
aerosols  include  particles  from  the  blasted  surroundings,  as 
well  as  from  the  explosives  themselves.  Depending  on  their 
experiences,  people  associate  these  odors  with  battle,  accidents, 
demolition,  or  fireworks.  They  could  be  simulated  to  some 

degree,  but  not  from  the  real  thing.  I  think  it  could  be  done. 
Other  forms  of  combustion 

Combustion  of  substances  containing  carbon  and  hydrogen 
always  presents  a  toxicity  problem  because  the  oxygen  reacts 
preferentially  with  the  hydrogen,  increasing  the  C-to-H  ratio, 
which  leads  to  the  formation  of  benzenoid  and  ultimately  the 
carcinogenic  polycyclic  aromatic  hydrocarbons.  But  one  doesn't 
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have  to  go  that  far  to  get  a  burnt  odor.  I  did  use  oil  of  cade 
(Juniper  tar)  for  the  diesel  burnt  note,  but  that  may  have  been 
pushing  it  a  bit  too  far  toward  toxicity.  Caramelized  odors  are 
on  the  early  chemical  pathway  to  tarry  and  then  burnt  odors,  and 
they  may  be  reasonable  substances  to  use.  There  are  also 
synthetic  hickory-type  odorants  for  barbecue  applications,  and  it 
would  be  good  to  check  these  out. 

For  burning  plastic  and  insulation,  it  is  essential  to  avoid 
any  source  that  contains  chlorine  in  any  form,  because  combustion 
of  such  materials  always  leads  to  at  least  some  dioxins  and  furan 
derivatives,  which  are  toxic.  Polyvinyl  chloride  depolymerizes 
to  vinyl  chloride,  a  known  carcinogen.  The  acridity  of  these 
odors  can  be  simulated  by  other  reference  compounds. 

I  don't  have  experience  with  odors  associated  with  surgical 
processes,  but  I  would  want  to  check  first  to  establish  how  much 
of  the  sensory  impact  in  such  environments  comes  from  the 
anesthetics,  and  how  much  from  the  sterilizing  and  cleaning 
compounds.  Cauterization  gives  a  burning  flesh  odor.  Have  these 
effects  in  hospital  environments  been  established? 

Conclusion 

I  think  the  chances  are  fairly  good,  but  by  no  means 
certain,  that  your  objectives  can  be  accomplished  at  reasonable 
cost.  It  would  be  prudent  to  carry  out  some  of  the  preliminary 
tests  I  have  suggested  before  large  investments  of  money  and 
effort  are  committed. 

Amos  Turk 
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Non-odor  aspects 


The  generation  of  aerosols  such  as  those  that  accompany 
diesel  exhaust  would  add,  in  my  opinion,  an  unnecessary 
complication,  especially  at  the  outset,  when  there  will  otherwise 
be  complications  enough.  Remember,  diesel  engines  are  not 
necessarily  smoky,  and  should  not  be  when  they  are  well  tuned. 
Explosions  and  uncontrolled  fires  from  organic  matter  are  always 
smoky,  and  range  from  white  to  black,  depending  on  moisture  and 
carbon  contents.  If  smoke  is  to  be  simulated,  white  is 
preferable  to  black,  which  generally  includes  toxins.  Hold  it 
for  later. 

The  role  of  moisture  is  interesting  and  not  generally 
understood.  Tests  conducted  years  ago  at  the  then  Cleveland  labs 
of  ASHVE  (predecessor  of  ASHRAE)  showed  that,  other  things  being 
equal,  atmospheric  humidity  tends  to  decrease  odor  intensity,  but 
that  things  are  rarely  equal.  Much  more  common  is  the  effect 
that  moisture  has  on  displacing  adsorbed  odors  from  furnishings, 
coatings,  and  other  materials,  so  that  intensity  actually  rises 
when  humidity  does. 
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Design  of  an  Optical  Orientation  Sensor 


1.  System  Specifications 

(Discussed  with  Myron  Krueger  ;  Formalized  by  Jannick  Rolland) 

Given  a  40  feet  square  room  with  Light  Emitting  Diodes  (LEDs)  at  various  locations  (e.g.  on  the 
head  of  a  person),  design  an  optical  system  that  can  measure  the  orientation  in  azimuth  and 
elevation  of  an  LED  in  the  field  of  view  (FOV)  of  the  optical  system.  Moreover,  as  a  special 
requirement  the  system  must  be  designed  to  use  a  linear  detector  array  (ID)  instead  of  a  two 
dimensional  (2D)  array.  Cost,  speed,  and  resolution  are  issues  in  favor  of  a  ID  array.  Finally, 
the  optical  system  will  preferably  be  able  to  image  a  90  degree  FOV  horizontally  and  a  20  degree 
FOV  vertically  while  the  same  optical  system  should  be  used  if  possible  whether  the  ID  array  is 
positioned  horizontally  or  vertically.  The  final  FOV  and  ratio  of  the  x-component  to  the  y- 
conaponent  should  be  specified  taking  into  account  a  required  resolution  for  the  overall  system. 

•  The  ID  detector  array  will  be  a  6K  DALSA  sensor  with  10  um  pixels  (see  attached 
documentation) 

•  The  source  will  be  a  high-power  GaAIAs  illuminator  (model  OD-100)  with  a  peak  emission 
of  880  nm;  the  spectral  characteristic  at  20%  spectral  output  is  about  150  nm  (810nm- 
960nm)  (see  attached  documentation).  The  physical  size  of  the  LED  source  is  1  mm  x  Imm. 
The  radiant  intensity  of  the  source  depends  on  its  mode  of  operation,  continuous  or  pulsed. 
The  lower  bound  will  be  used  in  the  calculation  of  the  SNR  in  the  last  section  of  this  report. 
LED  model  OD-666  was  originally  chosen  and  this  study  influenced  the  decision  to  work 
with  the  OD-100  model  which  has  a  smaller  source  area. 

2.  Overall  Design  Approach 

Such  specifications  can  be  achieved  using  a  double  stage  anamorphic  optical  system.'  The  first 
stage  will  consists  of  an  anamorphic  fisheye  lens  which  will  allow  imaging  of  a  90  degree  x  20 
degree  FOV  onto  a  square  virtual  detector  array  also  known  as  an  intermediary  image  plane.  No 
real  detector  is  needed  here  since  the  image  will  be  used  by  the  second  stage  and  only  a  final 
detector  is  required.  The  second  stage  will  be  an  anamorphical  imaging  system  that  will  map  the 
square  intermediary  image  onto  a  ID  linear  CCD  array. 

Two  questions  arose: 

•  Can  a  fisheye  lens  be  designed  to  be  anamorphic  ?  (a  novel  approach  is  presented) 

•  Will  such  a  mapping  allow  enough  light  collection  from  one  LED  on  the  ID  CCD  array? 
(Radiometry  calculations  indicate  that  the  S/N  ratio  is  >1  in  all  cases  considered) 

This  report  discusses  the  optical  system  design  approach  to  the  overall  system  (section  3  and  4), 
the  resolution  and  field  of  view  tradeoff  (section  5),  the  impact  of  the  size  of  the  image  of  the 


'  Anamorphic  here  means  that  the  optical  system  yields  different  magnifications  in  the  two  orthogonal  x  and  y 
directions.  An  anamorphic  system  usually  require  cylindrical  optics  of  one  sort  or  another. 
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LED  on  detection  (section  6),  and  presents  the  light  collection  calculations  for  feasibility  of  the 
system  (section  7).  This  report  does  not  include  the  final  lens  design;  It  includes,  however, 
some  preliminary  optimization  of  the  anamorphic  fisheye  lens. 

The  study  reported  here  focuses  on  demonstration  of  feasibility.  We  show  a  case  study  assuming 
a  80  degree  FOV  in  x  and  a  20  degree  FOV  in  y.  There  is  no  fundametal  limit  to  go  from  80  to 
90  degree  FOV,  only  more  complex  optics  is  usually  required  as  the  FOV  increases  for  a  given 
image  quality.  The  80  degree  was  chosen  for  this  feasibility  study  because  the  fisheye  lens 
chosen  as  a  starting  point  and  shown  in  section  3  was  optimized  up  to  ±43.5  degrees.  Increasing 
the  FOV  by  only  a  couple  of  degrees  at  such  wide  angles  is  significant  and  not  necesary  in 
completing  this  first  phase  of  the  study.  To  insure  that  all  rays  were  imaged  with  high  quality  as 
a  starting  point,  we  decided  to  select  an  80  degree  FOV  (±40.0  degrees)  in  x.  Because  only  4 
cameras  covering  90  degree  each  and  placed  at  the  comers  of  a  room  would  be  ideal  for  the 
application  at  hand,  the  FOV  in  the  final  system  will  be  increased  from  80  to  90  degree  in  one 
dimension.  In  the  other  dimension,  while  20  degree  would  yield  higher  precision  of  tracking,  a 
larger  FOV  may  be  beneficial  to  cover  a  larger  tracking  volume  (e.g.  including  height  of 
children’s  head  instead  of  essentially  adults).  Some  considerations  of  the  trade-off  between  the 
application  requirement  and  the  engineering  requirements  still  need  to  be  made.  The  negineering 
requirements  are  given  here. 
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3.  Design  of  an  Anamorphic  Fisheye  Lens 


The  starting  point  for  the  design  was 
chosen  to  be  that  of  a  fisheye  lens 
that  we  carried  out  a  few  years  ago 
for  the  design  of  miniature  video 
cameras  for  a  video  see-through 
head-mounted  display  (Edwards, 
Rolland,  and  Keller,  1995).  This 
lens  consisted  of  two  single  front 
meniscus  lenses  and  two  doublet 
lenses  close  to  the  aperture  stop  of 
the  system  as  shown  in  Figiue  1 . 

This  lens  will  need  to  be  scaled  to 
the  proper  focal  length  for  the  design 
of  the  proposed  system. 


Figure  2.  On  axis  imaging  of  an  anamorphic  fisheye  lens  (a)  in  the  x 
meridional  plane  (b)  in  the  y  meridional  plane. 


We  demonstrate 
here  that  a  fisheye 
lens  can  be  made 
anamorphic  by 
insertion  of  an 
anamorphic  focal 
attachment  located 
appropriately 
within  the  system. 
The  attachment,  in 
this  case,  is 
positioned  between 
the  two  doublets  as 
shown  in  Figures 
2a  and  2b  which 
show  the  imaging 
of  a  point  on  axis 
through  the 
anamorphic  fisheye 
lens  for  an  x  and  y 
meridional  plan, 
respectively. 
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Figure  3  show  the 
imaging  of  the 
anamorphic  fisheye 
lens  for  a  point  off- 
axis  but  close  to  the 
axis  (small  field  of 
view)  (a)  and  (b) 
and  a  point  further 
off-axis  (c)  and  (d). 
No  attempt  to 
optimize  the  lens 
was  conducted  at 
this  point  in  time. 
These  figures  are 
only  meant  to 
illustrate  feasibility 
of  a  fisheye  lens 
with  anamorphic 
properties. 


While  the  field  of 
view  is  larger  in  x 
than  in  y,  the  image 
of  the  field  of  view 
by  the  fisheye  is 
square.  This  image 
will  be  fed  to  the 
next  stage  of  the 
optical  system. 

At  this  stage  of 
feasibility,  no 
attempt  was  made 
to  match  the  ratio 
of  the  X  and  y  field 
of  views  to 
required  values. 
Proper  scaling  will 
be  done  at  a  letter 
stage  of  the  design 
when  all  optical 
quantities  have 
been  determined. 

We  now  have 

demonstrated  that  we  can  transform  a  fisheye  lens  into  an  anamorphic  fisheye  lens  where  two 
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different  field  of  views  in  x  and  y  will  be  mapped  on  a  square  intermediary  image.  Latter  in  this 
report  we  shall  calculate  the  specification  parameters  of  the  overall  system.  These  parameter  will 
be  used  to  scale  the  fisheye  lens  appropriately.  The  anamorphic  lens  will  be  designed  to  be  part 
of  the  scaled  fisheye  lens  and  the  anamorphic  fisheye  lens  will  then  be  optimized  for  satisfactory 
performance  which  will  be  discussed  in  section  5. 

The  next  stage  of  the  design  is  to  take  the  square  intermediary  image  and  image  it  on  a  linear 
array.  Another  anamorphic  imaging  system  is  needed. 

4.  Anamorphic  Lens  From  The  Output  Of  The  Fisheye  Lens  To  The  CCD  Array. 


Such  a  mapping  as  illustrated  above  is  quite  unusual  in  optical  systems.  The  rationale  for  such  a 
mapping  is  to  be  able  to  use  a  ID  linear  array  sensor.  A  first  order  layout  of  a  system  which 
would  allow  such  a  mapping  is  given  in  Figures  5a  and  5b.  The  idea  is  to  create  an  astigmatic 
image  (property  that  one  usually  strives  to  eliminate).  This  can  be  accomplished  using 
cylindrical  optic  in  a  specific  assembly.  In  a  first  order  calculation  one  requires  only  two 
positive  cylindrical  lenses  satisfying  EFL2  or  F2  equal  2  EFL3or2F3  where  EFLofF 
denotes  the  focal  length  and  2  and  3  refer  to  Lens  2  and  Lens  3  shown  as  L2  and  L3  in  Figure  5. 

The  image  plane  of  the  fisheye  lens  (also  referred  to  as  the  intermediary  image  plane)  is  located 
at  the  anti-nodal  points  (i.e.  twice  the  focal  length)  of  Lens  3^.  At  the  anti-nodal  point  image  of 
Lens  3  is  an  image  of  the  x-dimension  of  the  intermediary  image  plane.  The  magnification  is  -1 


*  Antinodal  points  in  optical  systems  are  defined  as  the  optical  conjugates  of  angular  magnification  -1,  which  also 
correspond  to  a  linear  magnification  of  -1.  They  often  present  interesting  properties  of  symmetry  for  imaging. 
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so  the  size  of  the  detector  sets  the  overall  dimension  of  the  system.  The  detector  has  6000  pixels 
of  10  um  X  10  um,  thus  it  is  60  mm  long  (see  Figure  5a).  The  image  plane  of  the  fisheye  lens  is 
also  located  at  the  focal  point  of  Lens  2  and  is  therefore  imaged  to  infinity  after  Lens  2  in  Figure 
5b.  Lens  3  in  Figure  5b,  is  equivalent  to  a  plane  parallel  plate.  In  the  focal  plane  image  of  Lens 
2  is  an  image  of  the  exit  pupil  from  the  fisheye  lens.  The  height  h  of  the  exit  pupil  is  given  by 

EFL2 

h  = - p  , 

EFLl  i 

where  EFLl  and  EFL2  are  the  focal  lengths  of  Lens  1  and  Lens  2,  respectively;  p;  is  the  height 
of  the  exit  pupil  of  the  fisheye  lens.  Knowing  the  height  h  of  the  “exit  pupil”  at  the  final  image 
plane  (i.e.  the  detector  plane)  will  be  useful  in  the  calculation  of  energy  reaching  each  pixel  at  the 
detector  plane.  We  will  also  have  to  consider  the  width  of  the  line  due  to  the  finite  size  of  the 
LED. 


Fisheye  image 


FI  F2  F2 


Figure  5.  Paraxial  layout  of  the  anamorphic  optics  between  the 
image  plane  of  thefFisheye  lens  and  the  CCD  ID  detector:  a. 
along  linear  dimension  of  the  array ;  b.  perpendicular  to  the 
linear  dimension  of  the  array. 


Given  the  size  of  the 
detector  we  know  that  the 
image  size  of  the 
intermediary  image  plane  is 
60x60  mm.  This  allows 
specification  of  the  focal 
length  of  the  fisheye  lens 
given  the  FOVs  to  be 
imaged. 


For  FOVx  =  ±  40  degrees 
and  FOVy  =  ±  10  degrees 
we  can  calculate  the  focal 
lengths  EFLx  and  EFLy  of 
the  fisheye  lens  (y;  =  30 
mm). 


y 

EFL,  =  ^  =  42.97  mm 


171.89  mm 


(Fisheye  lens  specification) 


Note:  want  a  linear 
mapping  so  use  0  in  rad. 
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Given  a  FOVx  equal  to  four  times  FOVy,  the  magnifying  power  MP  of  the  afocal,  anamorphic 
attachment  is  4  and  the  ratio  of  the  aperture  stop  of  the  fisheye  lens  to  that  of  its  image  through 
the  afocal  attachment  is  4  as  well.  A  paraxial  raytrace  gives  the  values  of  34.75  mm  and  8.70 
mm  for  the  height  of  the  marginal  ray  at  surface  8  (the  aperture  stop)  and  surface  12  (the  back 
side  of  the  afocal  attachment,  respectively. 


The  lens  shown  in  Figure  6  has  been  optimized  to  satisfy  paraxial  properties,  such  as  FOVx 
equals  four  times  FOVy.  EFLx  =  42.972  mm  and  EFLy  =  171.887  nun  (EFL  stands  for  effective 
focal  length).  The  RMS  spot  size  at  various  points  in  the  FOV  are  currently  without  full 
optimization  :  on  axis:  0.22  mm  ;  FOVx  =  28  degree,  RMS  spot  size  =  0.35  mm  ;  FOVx  =40 
degree,  RMS  spot  size  =  0.61  mm ;  FOVy  =  7  degree,  RMS  spot  size  =  0.45  mm  ;  FOVy  =10 
degree,  RMS  spot  size  =  0.80  mm.  For  such  a  system  with  a  CCD  detector  where  the  centroid  of 
energy  needs  to  be  extracted  as  accurately  as  possible,  two  important  criteria  are  a  symmetrical 
spot  image  and  a  uniform  ullimination  over  the  image  of  the  LED.  To  yield  a  symmetrical 
image,  one  image  quality  rquirement  will  be  to  have  no  coma^  Currently,  image  quality  is 


’  Coma  is  one  of  the  third-order  optical  aberrations  in  optica!  systems.  Unlike  various  other  aberrations,  the  image 
spot  size  of  a  system  limited  by  coma  is  non  symmetrical  around  the  centroid  of  energy.  In  systems  relying  on  the 
centroid  of  energy  to  locate  a  point  in  the  FOV,  coma  should  be  eliminated.  Other  types  of  aberrations  (e.g. 
spherical  aberration,  defocus,  field  curvature,  astigmatism)  which  do  not  shift  the  centroid  of  energy  are  not  as 
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degraded  by  coma  and  further  optimization  of  the  lens  will  allow  to  satisfy  all  requirements, 
paraxial  and  symmetry  of  the  spot  size.  As  part  of  the  optimization,  we  will  also  aim  at 
decreasing  the  RMS  spot  size.  A  3  pixel  image  spot  size  is  desirable  to  get  good  centroid 
estimation  as  also  indicated  in  the  literature  on  stellar  tracking  systems  and  subpixel  resolution. 

Given  the  intermediary  image  formed  by  the  anamorphic  fisheye  lens,  the  next  stage  will  image 
the  square  intermediary  image  onto  a  ID  linear  CCD  array.  The  magnification  of  the  second 
stage  will  be  one.  We  need  to  calculate  the  length  of  the  astigmatic  line  formed  on  the  ID  CCD 
array.  The  principle  of  imaging  was  given  in  Figure  5,  assuming  that  the  exit  pupil  is  the  same  in 
X  and  y  location  for  the  fisheye  lens.  From  the  preliminary  design  of  the  first  stage,  however,  we 
also  note  that  the  exit  pupil  of  the  anamorphic  fisheye  lens  is  located  in  different  planes  for  the  x 
and  y  imaging  directions.  This  is  somewhat  of  a  complication.  The  solution  seems  to  lie  in 
observing  that  the  location  of  the  exit  pupil  for  the  x  imaging  which  is  currently  +800  mm,  can 
be  made  to  be  infinity  (which  we  will  do  in  the  next  stage  of  optimization)  and  that  of  the  pupil 
in  the  y  dimension  is  at  -385  mm.  So  by  using  a  cylindrical  lens  for  Lens  1,  we  can  put  the  exit 
pupil  at  infinity  in  both  directions.  Then  imaging  after  Lens  1  resumes  as  illustrated  in  Figure  5. 

Currently,  in  the  y  dimension,  the  diameter  of  the  exit  pupil  of  the  fisheye  is  equal  to  34  mm , 
and  its  distance  from  the  intermediary  image  is  385  mm  to  the  left.  For  the  design  of  the  second 
stage  we  shall  start  with  a  cylindrical  Lens  1  of  EFLly  =  385  mm  placed  close  to  the 
intermediary  image  as  shown  Figure  5.  The  pupil  will  then  be  at  infinity  after  the  intermediary 
image  for  both  the  x  and  y  dimension  (assuming  the  fisheye  lens  images  the  pupil  at  infinity  in 
the  X  dimension).  Then  depending  whether  we  want  to  image  the  x  or  y  diemension  on  the  linear 
array  we  flip  the  module  L2-L3  and  the  detector  90  degree.  There  is  no  need  to  have  two 
different  optical  systems  for  the  two  scenario  x  or  y  imaging.  If  we  want  to  image 
simultaneously  the  x  and  y  directions,  however,  we  may  consider  placing  a  half-silvered  mirror 
after  the  intermediary  image  foilwing  the  fisheye  imaging  and  have  in  each  path  the  same 
module  flipped  90  degree.  One  linear  area  in  each  path  would  also  be  used. 

Calculation  of  EFL  of  Lens  2  and  3  can  be  conducted  based  on  their  required  diameter  and 
assuming  that  lenses  are  open  at  F/2  or  less  for  fabrication  feasibility.  In  the  thin  lens 
approximation,  the  half-height  H  of  the  astigmatic  line  is  given  by 

h  =  0  .  f  2  ,  where  0  is  given  by  the  angle  subtended  by  the  exit  pupil  of  the  anamorphic 
fisheye  lens  (see  Figure  5b).  Indeed,  h„  =  0  .  f,  with  f,=  385  mm  ,and  h„=  17 
mm,  then  0  =  2.5  degree. 

Now  we  have  f  2  =  2  f  3  ;  and  from  the  design  of  the  first  stage  the  angle  of  the  marginal  ray  0„ 
at  the  last  surface  is  less  or  equal  to  0.04  rad  for  the  case  where  the  pupil  is  far  away  (approx  00). 
Given  that  the  exit  pupil  of  the  fisheye  is  almost  at  infinity  in  that  case,  we  can  estimate  the 
diameter  of  Lens  2  and  Lens  3  with 


critical.  Spherical  aberration  which  is  constant  over  the  FOV  may  be  desirable  to  enlarge  the  point  spread  functin  to 
a  3  pixel  spot  size  in  the  detector  plane. 
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<j)  =  size  of  the  intermediary  image  +  2  f  2  .  0„  =  60  +  2  .  f  2  .  0.04  s  60  mm 

let’s  assume  f  2  is  open  at  F/2  ,  then  ^  also  equal  f  2/2.  Therefore  f  2  =  2^  =  120  mm  ; 
Consequently  f  3  =  f  2/2  =60  mm.  This  yields  a  Lens  3  open  at  F/1 ,  which  is  the  maximum  one 
would  want  to  work  with.  In  the  design  stage,  depending  on  the  required  image  quality,  we  may 
have  to  assume  Lens  2  open  at  F/3  or  F/4,  which  will  increase  the  length  of  the  optical  system. 

Estimated  minimum  lens  of  the  optical  system  for  a  60  mm  linear  detector  array  is 

Total  Minimum  Length  =  Fisheye  lens  Length  +  2  f  2  =  (776+240)  mm  =1016  mm  s  1  m 

Options  for  decreasing  the  overall  length  are  either  choosing  a  smaller  detector  or  decreasing  the 
ratio  of  FOVx  to  FOVy. 

The  length  of  the  line  in  the  plane  of  the  detector  array  in  a  direction  perpendicular  to  the  array  is 
given  by  h  =  (34/2)  *  (120/385)  =  5.3  mm  if  the  large  field  of  view  FOVx  is  imaged  on  the 
linear  array.  In  that  case,  the  pupil  in  the  perpendicular  dimension  (along  the  y  dimension)  is 
what  matters  in  terms  of  the  length  of  the  line  perpendicularly  to  the  array.  This  is  relatively 
small.  For  the  length  of  the  line  if  the  lens  is  used  in  the  other  direction  where  the  small  field  of 
view  (FOVx)  is  imaged  on  the  linear  dectector  array,  what  matters  is  the  angle  subtended  by  the 
pupil  in  the  x  direction  from  L2.  Half  the  length  of  the  line  is  then  given  by  the  product  of  the 
angle  subtended  by  the  pupil  from  L2  (estimated  to  be  0.17  rad)  times  the  focal  length  of  L2 
(120  mm);  the  length  of  the  line  is  then  estimated  to  be  42  mm.  Given  these  estimated  values  of 
the  lines  perpendicular  to  the  array  in  the  two  configuration  of  imaging,  a  calculation  of  energy 
transfer  through  the  system  allows  to  estimate  whether  enough  light  will  be  sensed  by  the 
detector  array.  This  is  expanded  in  the  next  section. 

5.  Discussion  of  field  of  view  and  resolution  trade-off 

Let’s  assume  a  linear  mapping  of  the  field  of  view  on  the  linear  array  detector.  In  other  words,  y 
equal  (f .  0y )  if  the  linear  array  is  along  the  y  dimension  (respectively  x  =  (f .  0^ )  if  the  linear 
array  is  along  the  x  dimension).  We  are  close  to  this  mapping  right  now  and  will  optimize  the 
system  to  satisfy  this  mapping  exactly  during  full  system  optimization. 

Given  a  FOVx  equal  to  90  degree  (±45  degree),  and  a  detector  size  of  60  mm  (±30  mm),  this  set 
an  overall  EFLx  of  (30/0.7854)  equal  38.22  mm.  Similarly  EFLy  equal  (30/  0.1745)  171.89 
mm. 


Assuming  that  we  can  resolve  the  centroid  of  energy  of  the  detected  LED  within  the  10  um 
resolution  of  the  detector  array  (subpixel  resolution  can  in  fact  be  achieved) ,  the  variation  in 
angle  A0  corresponding  to  a  Ay  (let’s  say)  of  10  um  is  given  by  A0  =  Ay  /  f  ,  which  yields  A0y  = 
0.01  /  171.89  =  0.2  arc  minute.  In  the  x-direction  we  have,  A0x  =  0.01  /  38.22  =  0.9  arc 
minute.  This  simple  calculation  indicates  that  even  in  the  worse  case  scenario  of  pixel  resolution 
(instead  of  subpixel  resolution),  precision  of  tracking  is  fairly  high  (less  than  1  arc  minute).  We 


Appendix  D  -  89 


September  25, 1998 


Jannick  P.  Rolland 


remmend  that  a  complete  analysis  be  carried  out  to  determine  what  angular  resolution  is  required 
ofr  the  y  dimension.  Overspecifying  the  system  will  increase  the  full  design  cost  and  perhaps  the 
cost  of  the  prototype  as  well. 


6.  Considerations  of  imaging  of  an  “extended”  light  source 

The  imaging  range  for  the  light  sources  spans  1.5  m  to  15  m  from  the  optical  system.  At  near 
range,  the  source  will  subtend  the  largest  visual  angle.  We  estimate  here  the  size  of  the  image  of 
the  LED  at  the  extreme  of  the  range.  The  calculation  is  based  on  a  light  source  of  physical 
dimension  1  mm  x  1  mm  (we  have  decided  upon  the  OD-100  ;  this  calculation  influenced  a 
change  from  the  OD-66  to  that  of  OD-100).  Given  the  largest  dimension  of  1.4  mm  ( the  pixel 
diagonal),  the  size  of  the  image  of  the  LED  through  the  optical  system  is  summarized  in  the  table 
below: 


size  of  the  image  of  the  LED 
(using  the  1 .4  mm) 

x=  1.5  m 

x=  15  m 

imaging  of  90  degree  on  a  60  mm  detector 
(f  =38.22  mm) 

0.04  mm  or  3.7pixel 
(x’=39.2  mm) 

0.004  mm  or  0.37  pixel 
(x’=38.3  mm) 

imaging  of  20  degree  on  a  60  mm  detector 
(f=  171.89  mm) 

0.18  mm  or  18  pixels 
(x’=194.14mm) 

0.05  mm  or  1 .8  pixels 
(x’=l 73.88  mm) 

In  the  table  the  image  size  is  given  by  y  = 
principles  by 

x' 

and  9  is  given  by  the  ratio  of  the  source  size  1.4  mm  divided  by  the  distance  x.  x  is  the  distance 
of  the  LED  to  the  object  principal  plane  of  the  optical  system  and  f  is  the  focal  length  in  the 
corresponding  dimension  x  or  y.  In  the  above  formula  x  is  negative  and  f  is  positive. 

Thus,  within  the  working  volume,  the  image  of  the  LED  on  the  detector  varies  from  0.37  pixel  to 
18  pixels.  Increasing  the  FOV  in  y  will  help  decreasing  the  size  of  the  LED  on  the  detector 
plane.  The  challenge  'with  having  a  large  image  of  the  LED  is  to  insure  uniformity  over  the 
overall  size  for  precise  detection  of  the  centroid.  Also,  some  edge  effects  will  have  to  be  taken 
into  account.  If  18  pixels  is  too  large  to  conserve  imiformity,  a  larger  FOVy  may  be  considered. 

In  the  cases  where  the  image  is  less  than  3  pixels,  the  point  spread  function  will  be  adjusted  to 
cover  at  least  a  3  pixel  area.  This  will  be  achieved  by  allowing  some  remaining  spherical 
aberration,  aberration  that  yields  symmetrical  spot  size  and  invariance  over  the  FOV.  Coma  will 
be  eliminated  and  field  curvature  will  be  minimized. 


0  x’,  where  x’  is  given  from  first  order  optics 
xf 

X  +  f 
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7.  Flux  Detection  Calculations 

7.1  Some  fundamentals  of  detector  devices 

A  detector  is  characterized  by  its  responsivity  R(X,  f)  where  X,  is  the  wavelength  of  the  light  and 
f  is  the  electrical  frequency  of  the  modulated  signal  or  chopping  frequency.  Usually  manufactors 
report  values  of  R  when  measured  with  standard  test  sources  (500K  for  deep  infrared,  2856K  for 
visible  and  near  infrared  are  typical  sources). 

An  important  quantity  is  the  Noise  Equivalent  Power  (NEP,  measured  in  Watts)  defined  as  the 
required  source  power  to  yield  a  S/N  ratio  of  1.  NEP(X,  f)  as  well.  NEP=  (j)g/  (S/N)  ; 

Given  ({)e ,  a  good  detector  has  a  high  S/N  ratio,  therefore  a  low  NEP.  Given  a  detector  NEP  or 
an  equivalent,  and  given  (t)g  we  will  use  this  formula  to  derive  the  expression  for  the  S/N  ratio. 
In  some  case,  the  Noise  equivalent  exposure  is  given  instead  of  the  NEP.  More  on  this  and  the 
relationship  of  NEE  to  NEP  is  given  thereafter. 

Also,  an  overrriding  principle  is  that  given  the  integration  time  tj  of  a  pixel  element  (which  is 
equal  to  the  readout  time  in  our  case),  the  noise  equivalent  bandwidth  is  given  by  l/2td.  Also  the 
NEE  scales  as  the  square  root  of  the  noise  equivalent  bandwidth. 


7.2  Application  to  the  orientation  sensor 

This  calculation  first  assumes  that  the  width  of  the  image  of  the  LED  is  one  pixel  wide  and  off 
axis  at  some  angle  of  view  01.  The  led  off  axis  is  also  seen  from  the  pupil  under  an  apparent 
area.  The  actual  width  will  be  accounted  for  in  the  final  figure  of  merit  for  the  SNR.  The  final 
result  will  be  divided  by  the  actual  spread  of  the  source  in  number  of  pixels  that  is  a  maximum  of 
18  according  to  the  calculation  in  Section  6. 


<l>pixel  ^  •  L, 


source  •  ■^source  • 


'lens 


^ pixel  =  cos^JQ,=(^j(^j  coses)Al  cosei)/(xlcose^^2  =  {L^A^Ai  cos^  0,)/x^ 

where  we  set  0s=01  by  geometry;  Let’s  formulate  the  expression  for  the  pixel-SNR  based  on 
voltage  outputs: 

Vsignai "  •  N  ^  radiometry  theory) 

where: 

•  Ry  is  the  responsivity  (v  refers  to  a  voltage  output) 

•  Lsouf„  is  the  luminance  of  the  source  in  W/m^.sr 

•  ^source  is  the  area  of  the  source  (m^ )  (we  are  assuming  here  the 
extended  source) 

•  Q,e„s  (sr)  is  the  solid  angle  subtended  by  the  lens  at  the  source 

=  A,e„/x^  on  axis  where  x  is  the  distance  source  to  lens. 


more  general  case  of  an 

|;^=^^-Qlens - 

Asource 
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•  N  is  the  number  of  equivalent  pixel  elements  in  the  astigmatic  line  formed  in  the  image 
plane,  given  that  the  system  performs  point  to  line  imaging.  This  factor  allows  taking  into 
account  the  fact  that  energy  in  the  line  outside  of  the  pixel  detector  element  is  lost.  Still, 
here  the  width  of  the  line  is  assumed  to  be  one  pixel  wide. 

Thus,  =  T.  .  A„^cos''0s  /  (Nx^)  , 

12 

where  R^ .  =  jRv(X)L(^)d  ^  where  [Xl ,  X2  ]  represents  the  intersection  of  the  source 

II 

spectral  range  with  that  of  the  detector. 


Therefore  the  SNR  is  given  by 


SNR=^ 

Vnoise 


T.Rv  COS^  6  LsourceAsourceAlens 
VnoiseNx^ 


where  is  the  rms  noise  for  one  pixel.  Also,  by  definition,  for  one  pixel  NEP  =  V„oise  /  Rv 

Instead  of  providing  the  NEP  (W),  the  manufactor  specifies  a  quantity  called  NEE  to  denote  the 
noise  equivalent  exposure  in  pJ/cm^  The  NEE  is  related  to  the  NEP  as 

NEE  (pJ/  cm^  )=  NEP,  (pW/  cm^)  .  t„  (sec) 

(NEP,  is  used  here  for  conversion  units  purpose) 

andNEP(pW)=NEP,(pW/cm^)  .  Apixel(cm^) 

where  Apixel  is  the  area  of  a  pixel  and  tj  is  the  pixel  integration  time,  in  our  case  3000pixels/20 
Mhz  =  1 50  us  (note:  this  is  based  on  a  readout  time  of  20Mhz  and  the  fact  that  2  pixels  are  read 
simultaneously  during  each  readout).  An  important  point  is  that  the  NEE  is  often  given  for  a 
certain  bandwidth,  and  a  correction  factor  will  need  to  be  computed  to  account  for  the  working 
bandwidth. 


SMP- 

COS^0  LsourceAsource  Aens 

NEEApixel 

Nx^ 

Note:  the  NEE  is  likely  given  for  peak  wavelength.  We  must  theoretically 

Multiply  by  the  ratio  of  the  responsivity  at  the  wavelength;  the  SNR  decreases  in  proportion. 
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Numerical  Application: 

The  minimum  value  of  integration  time  was  estimated  to  be  1 50  us.  The  calculation  is  based 
on  a  20MHZ  readout  time  knowing  that  a  pair  of  pixels  is  read  simultaneously.  Therefore,  the 
integration  time  is  given  by  3000/20MHZ  or  150us. 

NEE  =  25  pJ/  cm^  according  to  spec  sheet  ;  this  is  however  for  a  7.5Mhz  instead  of  our  actual 
20Mhz  bandwidth.  20  MHz  corresponds  to  an  integration  time  of  1 50us  which  yields  a  noise 
equivalent  bandwidth  of  3.3Khz.  7.5Mhz  corresponds  to  an  integration  time  of  400us  which 
yields  a  noise  equivalent  bandwidth  of  1 .25KHz.  So  the  effective  NEE  should  be  25  x  1 .6  = 
40pJ/cm^  where  1 .6  is  the  ratio  of  square  root  of  3.3KHz  to  the  square  root  of  1 .25KHz. 

Apixel=  10  um  X  10  um  =  0.001  cm  X  0.001  cm  =  10"^  cm^ 

Radiant  intensity  of  the  source  le  (mW/sr)  =  Lsource  (mW/sr.  cm^)  .  Asource  (cm^)  -  From  the 
specification  for  the  OD-100,  the  minimum  power  is  lOOmW.  What  we  may  be  able  to  drive  out 
of  the  LED  may  be  higher  by  a  factor  of  2  to  5  but  we  will  use  the  most  pessimistic  power  value 
in  the  estimation  of  the  SNR. 

Alens  =  ndV4  with  d=l  .5  cm  in  our  case  (d  is  the  entrance  pupil  size  of  the  fisheye  lens). 

N  for  the  x  and  y  directions  are  given  by  (line  length  (um)  /  pixel  size  (um))  equal  (5300  um  / 
10  um  =  530 )  and  (42  000  um  /lO  um=  4200) ,  respectively. 

In  the  worse  case,  x  =  15  m  =  1500  cm.  For  T=1  ;  angle  0  is  zero  (led  on  axis) 

SNR  =  —  18  10“ 

N 


Thus  in  the  worse  scenario  where  N  equals  4200  equivalent  pixels,  the  pixel-SNR  is  estimated  to 
be  equal  to  43  »1.  This  assumes  a  pulsed  source  of  60mW/sr  and  a  150us  integration  time. 
The  pulsed  source  may  indeed  be  of  higher  intensity  since  we  used  the  most  pessimistic  case  of 
the  power  from  a  continuous  source.  In  the  case  where  the  light  source  spread  over  18  pixels,  the 
SNR=  43/18  =  2.4  >1.  In  a  more  typical  case  where  the  light  source  will  be  spread  at  least 
over  3pixels,  the  SNR  is  estimated  to  be  around  14.  In  all  cases  considered,  the  SNR  >1 . 

Note:  we  computed  SNR  for  various  sensor  DALSA IL-C8  (bOOOpix  lOum),  DALSA IL-C7 
(4096  pix  7um  at  either  20  or  30  Mhz);  EG&G  (4096pix  7um  20  or  30MHz  or  8192pix  7um 
20Mhz);  did  not  have  the  responsivity  for  the  Sony  Ilx503A  2048  pix  or  Kodak  KLI-5001 
5000pix.  The  SNR  was  found  >  1  for  all  cases. 
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Tecittlirsi  Pei»rt  of  A  1:1  AMmorpldc  Relay  Lens  SyilMi 


l.fKmieiir. 

IJ,  Prfiftirtf  <if  Og  tytiUm 

OiviM  a  rDom  witb  Light  Eoyttiag  EModea  (LED's)  at  various  location  on  an  objact  (e.g. 
on  the  head  of  a  person),  an  optical  traddi^  systen  mist  be  able  to  measuie  the  poihion 
and  the  orientation  in  azimuA  and  elevation  of  the  oh^  in  the  field  at  view  (POV)  of 
the  optical  system.  TWo  perhaps  unique  fisatures  of  the  system  are  die  use  of  a  Uuear 
detector  array  (ID)  instead  of  a  two  dimmsional  (2D)  anay,  and  an  anamorphic  field  of 
view  (POV)  in  the  ratio  of  3.  Cost,  ^leed,  and  resolution  are  issues  in  favor  of  a  ID  array. 


U.1.1 


Source 

Reid  of  view  (POV) 

90  degrees  horizontally  and  30  degrees 
vertically 

Numerical  Aperture 

To  be  assigned  for  sufficient  stgnal/noise  at 
the  output 

Bulk 

As  compact  as  possible 

Detector 

IDCCD  sensor  array  with  lOum  pixels 

Cost 

Less  than  $10,000 

LIJ.  OvnnI  Mfn  Approicb. 

The  system  was  originatty  conceived  using  a  double  stage  anammrpbic  optical  syteem 
(anamorphic  here  means  that  the  qrtical  syteem  yields  diffinem  magi^cations  in  die  two 
orthogonal  x  and  y  dhectioos.  An  anamorphic  system  typically  requires  cylindrical  optics 
of  one  sort  or  another).  The  find  stage  cmsists  hi  an  anamonihic  fiiheye  lens  which 
allows  imaging  a  90  degree  x  30  degree  POV  onto  a  squared  array  intermediary  image 
plane.  The  Intermediary  image  is  then  imaged  by  the  second  anamorphic  relay  lens  onto 
the  linear  detector  array.  The  main  drawbadt  of  the  system  as  conceived  is  its  cost.  The 
idea  of  a  new  design  ibr  the  second  stage  emerged  in  order  to  make  the  system  more  cost 
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eiTeclive.  Tbe  liyout  oi  tius  second  stage  as  wen  as  its  testing  on  an  (^ical  bench  is  now 
pfeseoted. 

1*2.  A  MW  for  the  sec6iid  sU^. 

If  we  look  at  a  schematic  r^msentation  of  the  hinctionlng  of  the  second  anamorphic 
stage,  it  seems  sinqile:  a  square  is  transCmned  in  a  line  (Fig.  1).  The  LEDs  are 
sequential  fired  so  that  the  position  of  one  L£D  at  a  time  is  coo^Nited. 


L  Scbemaik  RpRscMaiun  c^itK  second 


In  the  fbrmnr  relay  lens  the  ooe-to’one  imBging  was  achieved  sncoestfiiUy  to 
ipecificatioa  by  using  a  set  of  refractive  lenses  whereas  conceptually  a  simple  mirror 
should  he  able  to  perform  this  mapping.  Actuahy,  we  use  only  a  wimDr,  tiie  image 
which  conesponds  to  this  one-to-one  mtqiping  is  virtual  and  symmetrical  with  reflect  to 
tim  mkior  (Pig.2a).  So  came  up  the  idea  of  nsiitg  a  letroieflective  surface  hi  corobinatioo 
with  a  semitranspaieiit  plate  to  make  rays  cooueige  toward  the  virtual  image  given  by  a 
minor  and  to  make  the  image  real  (Fig.  2h).  At  the  output  of  the  fim  stage  the  system 
was  tehceoiric  setting  the  chief  rays  parallel  to  the  optical  axis. 
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fISWt  !•.  VirtMl  napfiiBg  with  only  a 


ft«d  Image 


flpme  QM4o4me  mapiilag  ibaoks  to  a 

letraKflecdve  Brtux  ami  a  fccainq^^ 


TheieftMe.  wc  are  able  to  do  a  one-to^ne  maj^ing  by  simply  using  a  letroreflective 
sur^.  lo  the  other  dimension,  must  add  cyiindrieal  fottowang  the  beanspUttar 
in  order  to  create  a  toe  in  the  image  plane.  The  image  plane  becomes  then  ccM||ogine  to 
the  exit  pupil  of  the  first  optical  stage  in  (me  dimension,  that  of  the  line.  The  system  could 
then  be  presented  according  to  Fig.  3  in  which  Hie  two  dUnensiottS  are  rq[ifesenied:  in  one 
dimension  we  have  a  one-UHMie  mapping  and  in  the  other  we  have  a  toe  image. 
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Stirli  8  lyitem  U  quke  i^mplr  coniMirH}  to  the  fonaer  system  usiiif  refractive  o|4ics  aq^ 
Ae  critical  componem  to  its  woricing  is  the  quality  of  the  letroreflective  sheet.  The  next 
step  is  to  design  this  system  so  that  it  will  be  at  compact  as  possible  and  to  test  its  imafe 
quality  on  the  bench.  We  shall  proceed  first  with  a  first  order  optical  layout  and  then 
optimize  the  optics  using  a  lens  design  software;  COPE  V  was  used  in  our  case. 

X  nrU  Order  l49Wl<> 

The  first  order  calculatioos  win  allow  us  to  ft)re8ee  the  apace  between  the  syoem's 
elements  to  make  h  as  compact  as  possible,  it  wiB  also  aUow  provide  a  startiog  pim 
r^lbrCXlDEV. 

First  of  al  let  iM  have  a  larger  diagram  to  huiDduce  the  nowions  we  ate  going  to  uae 
(Fig.  4). 
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ni»«  4  Nolaikm  far  Ibe  Ont  crder  layout. 

The  knowledge  of  the  positkNi  of  A’  and  B*  allows  us  to  constreliit  die  diameter  of  the 
cylindiipal  lens.  The  focal  length  f  of  this  lens  must  respect  the  relatioiiship  fs  D-D*.  The 
system  is  then  solved  for  xA*.  xB',  yA\  yB';  The  intermediary  results  xA,  xB,  yA,  yB  are 
also  computed  to  constraiot  the  beamsfdkter  dbuneter. 
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Tile  ottject  and  tbe  imaf»  aie  symmetrical  with  respect  to  the  beamspUtier.  In  our 
ifplipation,  the  object  wiH  be  1 14-milUiiieter  square.  We  can  choose  p  and  compute  the 
other  parameters  in  order  10  see  if  the  system  yields  a  valid  sohnioiL 

We  ke^  in  mind  that  the  system  must  he  as  compact  as  posshde  using  off-the-shelf 
conqKioents.  With  a  single  lens,  the  system  has  a  lot  of  constraints; 

•  We  cannot  have  a  small  focal  length  (to  limii  the  bulk)  and  a  latpe  diameter  because 
the  f-aumber(«t/diaaieter)ofthe  lens  must  be  greater  than  1. 

e  D  and  D*  numt  satisfied  f  «  P-P' 

•  D'  and  the  diameter  of  the  lens  must  be  so  that  the  lens  doesnl  collide  the 
beamsplitter. 

•  The  dimensioas  of  the  lens  must  be  sufficient  to  coflect  all  the  light  beam. 

if  we  look  again  in  the  catalogs,  we  notice  that  the  larger  dhneaiion  for  a  cylindrical  lens 
is  60*S0mm  and  the  minimal  focal  length  for  such  a  lens  is  IQOmm.  We  could  put  the 
lens  for  away  from  the  beamsplitter,  but  then  we  have  no  chance  to  have  a  compact 
system.  As  a  Sfhiuii  wide  fens  is  enough  to  receive  the  image  spot  ^st  idler  the 
beamsplitter,  we  can  think  about  using  this  kind  of  lens.  The  problem  is  that  a  100mm 
focal  length  imposes  puitiog  the  ohjiect  at  least  at  150mm  from  the  beamsplitter, 
moreover  these  calculation  are  very  approximate  as  we  are  not  taking  into  account  the 
thickness  of  the  elements,  so  that  real  dimensions  can  he  for  bigger.  That’s  why  we  must 
keep  a  small  focal  length. 

We  have  higliiighted  rapkUy  that  with  the  lenses  available  in  the  catalog!,  tbe  system  was 
overconstrainted  and  no  sotutkm  appeared.  Then  we  must  reconsider  tbe  structure  of  our 
system. 
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2^  ieareli  lir  a  NliitioM  iisilig  two  leiiie& 

For  thin  lenses  the  power  C  resulting  from  puttmg  two  lenses  (power  Cl  and  C2)  together 
is  OC14C2.  If  we  consider  the  focal  lengths  this  relationship  becomes: 

f  r,  h 

The  main  point  is  that  we  can  make  a  50mm-focal*length  lens  with  two  lOOronhfocal* 
length  lenses.  An  other  advantage  for  two  lenses  is  that  geometrical  aberrations 
decrease  when  the  numb^  of  lenses  mcreases. 

3.  Optical  Peaigii  niider  CODE 

We  use  the  first  order  layout  as  a  starting  poim  to  the  optimization  under  CODE  V. 

3.1.  Viral  lirawtag. 

To  model  the  system  in  GIXMB  V,  we  decide  to  unfold  the  system  and  to  consider  h  only 
after  the  reflection  on  the  retroreflective  sheet  because  there  is  no  preprogrammed  auface 
m  CODE  V  that  could  model  such  a  sheet.  We  then  analyze  the  pan  of  the  system 
composed  of  the  beamsplitter  and  the  two  lenses.  We  define  the  object  as  a  virtual  object 
at  which  the  rays  will  aim.  So  instead  of  definmg  a  real  object  with  the  entire  system  we 
define  §  virtual  object  with  the  unfolded  system  As  a  matter  of  fiwt  tlds  virtual  object 
takes  the  place  of  die  virtual  image  given  by  a  single  mirror.  A  simplified  diagram  for 
this  unfolded  part  is  shown  in  Fig.  6. 
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III  (kit  uoiplified  drawing,  tke  image  and  ttie  object  m  auperiinpoied,  but  it  oeitainiy 
wiU  not  be  the  case  in  the  lealinodeiiiigbecauie  of  tbe  optical  elements*  ihicknesg  which 
are  going  to  introduce  a  ihift.  Moreover  for  the  moment  we  don't  tilt  the  beamsplhter 
because  we  only  want  to  know  if  ow  configuraiioa  is  vahd  or  not  Alter  mdering  data  in 
CODE  V.  the  syttem  in  the  two  dimensiotts  it  shown  in  Fig.  7.  On  this  drawing  we  can 
see  the  shift  between  the  object  and  the  inu^  and  we  can  notice  thtt  the  concqit  of  our 
system  is  *1raytraceable**:  There  is  a  one-to-one  hnaging  in  the  Y-dimension  and  a  line  is 
fonied  in  the  X-dimeiision.  if  we  zoom  in,  we  can  notice  that  the  image  is  out  of  focus 
and  h  leads  us  natuiaUy  to  the  optimizatimi  phase. 
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]JL  OptlmimtiaM  of  tlie  qrstem. 

3J.l.nnlAUTOMicraw 

We  now  optimize  die  system.  Tbe  result  of  this  first  optimization  a^iears  on  Figure9. 


NomoiBddiicat 


lippfv  t.  Clrat^  cf  ilie  flm  opiimizaiim  ft  2kMan  to 
(xm^tian  fiv  delociis  of  ilie  center  fieM 

(Ike  (i^  is  Ml  Kfneseaied) 

iut  now  we  must  think  about  tilting  the  plate  because  k  has  an  iinportwi  part  to  for 
the  poskioiiing  of  the  lenses.  Actually  we  can  notice  that  in  our  first  optunization  CODE 
V  placed  the  first  lens  in  a  position  that  won’t  exist  anymore  when  the  heamspliiter  is 

dM 

IW  oiiiiiai#«  aM  a»#*  of  ||e  lairt  apilem. 

The  final  tesult  of  the  opthnization  is  shown  in  Fig.l(l&l  1. 
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l%M«  IMll.  Drawiags  of  Me  fiB«l  syttem. 
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Hie  Optical  elements  dun  1  used  are  Oe  following: 

•  CyliDder  lenses:  MElXESGiUai^''^M^ 

•  Beanisplilter:  EDMUND  SCIEhmi^ 


ilfve  11  pMi  meet  of  die  iMi  systen 

8po|Oli§nMi< 

Die  spot  (ttagmns  are  obtained  by  tracing  a  grid  of  equally  spaced  rays  on  tbe  entiiipe 
piqdl  firooi  evfa  point  in  tim  object.  Die  distribution  on  the  image  surface  is  4Kn 
reproduced  at  a  correct  scale.  Die  inteiett  of  timse  diagrams  is  that  tbey  show  the 
geometrical  structure  of  the  image;  moreoMer  when  they  are  drawn  in  color  they  allow  to 
tee  the  chromatic  aherratkminsofw  as  each  color  corresponds  to  one  wavelength.  Fig.  13 
shows  the  spot  diagrams  Ibr  five  fields:  fOiOX  (9.8jQi,  (144)X  (QlM).  (14.14)  which 
correspond  to  the  normalized  fields  (0.0).  (a7.0),  (1,0).  (ai).  (1.1). 
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llpM  11.  Spot  DiagEws  for  Ike  fiMl 


We  can  notice  that  the  chforoatic  aberration  is  not  a  problem  for  our  apf^ation  because 
it  causes  no  spread  in  the  Y-direction.  As  it  was  previewed  the  quality  of  the  line  is  for 
much  better  for  a  field  point  that  betongs  to  the  YZ-plane,  where  as  for  those  which  move 
along  the  X-axis  we  see  some  defocus;  and  the  greater  the  field,  the  greater  the  defocus. 
But  the  quality  of  the  image  along  the  X-axis  is  not  imfxirtant.  ActuaHy  what  we  are  glad 
to  observe  is  that  there  is  no  importaiit  spread  along  the  Y-dhection  in  the  center  of  the 
fine.  But  we  must  admil  that  the  scale  of  these  spot  (tiagrams  is  not  appropriate.  Because 
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Mne  of  die  iniior  «lofig  the  Y*<lirectkm  has  a  lOuin  width,  we  conducted  further 
analysis.  Till  is  the  pinpoae  of  the  Detector  Energy. 

A14  INadMrBMfiyfGW). 

43DE  measures  the  relative  proportion  of  passable  rays  at  each  field.  The  convolution  in 
the  nominal  image  siir£Ke  with  a  detector  is  included.  We  specify  a  detector  that  is  Inm 
long  in  the  X-digctioo  and  Mhim  long  in  the  Y-dhecUon.  Then  QDE  adds  a  21*21 
di^lay  of  points  in  the  image  showii^  the  energy  on  the  detector  when  it  is  centered  at 
each  of  those  points.  We  specify  an  increment  for  this  21*21  grid  which  is  dOuin. 

The  results  am  shown  in  Figures  14a/bftl5a/hi  They  are  the  results  for  the  center  field 
and  the  comer  field.  We  can  see  that  for  the  center  field  the  line  is  perfect;  but  for  the 
comer  field  tbe  fine  is  spread  along  the  Y<diRction.  Fbrtunaiely.  we  can  notice  that  dm 
distributioo  along  the  Y*direction  is  symm^rical  widi  respect  to  the  center  Ihie.  So  we 
can  go  around  this  bhir  problem  and  we  can  capture  the  point  we  aie  hnemsled  in  thanks 
to  that  symmetry.  However  we  must  be  careftil  because  for  an  extmne  comer  point  the 
image  points  that  are  above  the  center  Ime  will  be  out  of  the  aensor  line.  Thm’s  why  we 
loie  a  little  part  of  the  object,  in  Figure  ISb.  we  can  see  that  die  sfuead  is  320ua  (the 
Increinent  we  have  chosen  the  grid  is  40um)  lo  that  we  are  able  to  capnne  13.6g0inni 
Instead  of  14niro  we  judged  acceptaMe. 
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3.2.5/.  ToltnMCM  Aial9«li» 

Tolerances  of  tlie  finished  design  are  also  needed  in  order  to  have  the  unht  bulk  at  a 
reasonable  cost.  In  our  case  we  have  focused  our  optimizatkHi  effort  on  having  ofi> 
shelves  elements,  so  this  analysis  is  going  to  ensure  us  that  tolerances  given  by  the 
manufocturers  are  sufficient.  We  measure  the  performance  change  by  ihlS  (Root  Mean 
Squared)  wavefront  error  or  MTF  (Modulation  Transfer  PMion).  No  sensitive 
tolmnces  were  observed. 

tocitartng  of  81m  obJect*a  iWwruidnnB 

We  now  increase  the  dimensions  of  the  object  from  28*281100  to  42*42mm  to  investigate 
other  field  of  view  requirements.  Of  course  such  a  modification  can  have  importaitt 
consequences  on  the  system  we  have  designed.  So  we  uxdc  the  latest  configuratioa  and 
increased  the  object’s  dimensions  in  order  to  fit  those  new  needs.  The  results  are  shown 
onng.l6. 


flgaNlih 
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Sq^tember  25. 1998 


f%*«  Ite  iKxtsiiBg  atfcct's  dhmsioa  Mdcet  v48ie(tiN 


Foituostely.  we  can  notice  tbat  vignettii^  occurs  oiriy  in  tlie  X-dimensioii,  ie.  t|8 
dinensioii  in  which  it  formed  the  line.  As  a  nutter  of  foct  if  we  lose  a  part  of  the  hghl  hi 
this  dimession,  it  is  not  really  important  because  we  are  focused  on  the  ceoter  of  the 
inM^  (the  IDCOD  atiay  will  he  p^ndicular  fo  foe  image  line);  so  losing  a  uanll  part 
of  the  line*sed|es  is  not  critical  On  the  contrary  we  are  sttisHed  to  see  fogt  we  collect  att 
the  Ught  in  the  other  dhneiisioo.  Then  our  system  is  stUl  valid  for  a  42*42mm  oiyect. 


3.3^.  MotWiBiQfllif  nNweflMilYc  mrfMee. 

The  next  chattel^  it  to  model  the  retroreflective  surfoce  because  CODE  V  does  not 
iochide  a  type  of  aurfoce  that  has  the  optical  property  to  send  a  ray  bach  on  itself.  We 
have  to  *1aiild”  this  surfoce  manually.  Moreover,  we  dK>  have  to  specify  to  CODE  V 
that  the  plate  we  had  « the  unfolded  part  is  in  fact  a  beanupHtfor. 


Appendix  D  - 1 11 


ScfMealKr2S.  1998 


A.3.L  A  tm  lolMtiBB  mdag  ^k>^^•89^Mell^iil  Smkm, 

The  idea  of  this  first  sohition  k  based  on  the  fiKt  thsi  CODE  V  traces  defiuib  rays  for 
each  fi^  of  view  and  time  are  the  tays  in  the  X-Z  plane  and  the  rays  hi  the  Y>Z  plane. 
For  these  rays,  we  have  to  identify  their  position  on  the  sufive  represeming  the 
letroreflective  dieet  and  we  then  have  to  send  them  back  on  themselves.  An  idea  is  to 
divide  a  surfiioe  in  small  square  patches  (2nim  squaied  for  example)  and  when  a  ray  hits 
one  of  these  smrfoces,  we  identify  which  one  it  is  and  we  orient  it  so  that  the  ray 
concerned  is  seat  bade  on  itself.  In  such  a  configuration,  Non>Se()ueati8l  (NS)  rai^s  ate 
perfectly  appropriate  not  only  for  modeling  the  beamsplitter  but  also  for  entering  all  the 
small  patches  which  compose  the  retnueflective  surface.  Ihe  results  we  get  are  pretty 
nice  and  we  can  see  die  whole  system  on  a  CODE  V*s  drawing  (Fig.  Ha). 


npm  tTw  View  of  *e  «lK4e  listen  asiqg  Nn 

Actually,  the  figure  above  is  only  a  view  in  the  X-Z  plane  and  a  perspective  view 
(F%.17b)  shows  diat  we  built  the  divided  surface  only  to  catch  the  rays  in  that  plane.  In 
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&meato2S.mi 

tlM,  tke  lays  |race4  by  CODE  V  in  the  other  plane  ut  terminated  after  toacMog  the  ftoqt 
ipiface  of  the  hramrpllttfr. 


npM*  Db.  Penpective  view  of  hie  whole  •yEtm  HsiBg  NSS 


The  next  figures  (Pig.  ISaftlftb)  show  what  we  gel  for  the  other  dtmeoiioii  in  which  we 
have  a  one-toKNie  oapphig. 

naarv  talk  View  of  the  wlKite  qvtcM  in  We  caha  iliHien^ 


SqNcaber25, 1998 


Hgy*  |l>.  Peryctivc  view  of  the  wbote  gystetp  rotated  to  see  the  modeliag  iB  Ifac  oUkt  rtiacnsioa 


X5X  A  Macro  NwdiolBfloa. 

CXX>E  V  alio  provides  the  ci^abUhy  to  wHt  our  own  programs  in  a  language  called 
Macro  Phis  in  order  to  allow  us  to  have  oser-deflned  variables,  arrays,  functions  and 
suitKes  in  our  system.  We  ate  especially  interested  in  the  Act  that  CXIDE  V  has  several 
user-defined  featiues  to  supplement  the  built-in  forms.  These  include  the  ability  of  the 
user  to  program  a  user-defined  surface  (two  forms,  UDS  and  UD2). 


We  then  propose  to  write  a  shoit  mKfo  usiqg  this  USE|tSUR2  function  which  can 
program  a  surfoce  that  has  the  property  to  send  a  ray  of  light  back  on  iiaelf.  Basically,  we 
have  to  model  a  surfoce  in  a  way  that  any  ray  hitting  it  encounters  a  surface  exactly 
perpendicular  to  k.  Thus,  if  the  surface  is  a  reflector,  all  rays  will  reflect  back  onto 
themselves. 
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SefilMilier25.199e 

Unfoftuiuttely,  this  oMcro  does  not  work  whh  the  emiie  system.  I  manage  lo  make  it 
perfectly  work  whh  the  iinfoldBd  system  bid  af^iareiitly,  this  macro  is  not  compiuihle 
with  NSS  ranges.  The  proMein  was  not  clearty  identifiable  but  when  we  add  some  NS 
surfKes  in  the  system,  the  macro  doesn’t  work  property  and  only  two  rays  are  cmnpuled 
entirely.  Nevertheless  it  works  quite  well  whh  the  unfolded  system  that  we  first  designed. 
Figures  19aftb  show  that  rays  coming  from  the  object  aee  sent  back  on  themselves  to 
frrnn  the  fine  image.  This  time,  instead  of  having  the  mys  aiming  at  die  vhttial  oigect,  we 
make  them  start  from  this  object  and  they  am  sent  back  thaidcs  to  the  UD2  aurfoce. 


Appendix  D  -- 115 


SquemberlS,  1998 


ill.  liiAbBidiineBskiialaMlper^iectivsvkwcf  riieiiscaftiKUD2aarftce 


Now  that  we  have  our  emke  system  designed  under  CODE  V  we  would  Mte  to 
concretely  verify  its  ftinctioiiing.  We  need  to  detemine  all  the  optical  and  mechanical 
elements  that  are  going  to  make  this  experimem  possible. 

4.  Optical  tad  tgchtmigri  hardwarg. 

4.1.  Opllol  elenicpHi. 

In  this  paragraph,  we  are  gorng  to  describe  all  the  qitical  elemenis  involved  in  the 
fxperimeot  We  shall  also  include  the  characteristics  of  the  LED  here. 

il.t1liihaimipillw. 

When  we  designed  the  system  under  CX)DE  V  we  realized  that  the  beamsplliier  would 
hauetobebifierthanwhat  we  found  usually  in  the  catalogs.  We  could  have  asked  for  a 
custondzedbeaimidftter  but  cost  was  an  issue  mtihs  prototype  stage.  The  material  used 
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for  Item  is  the  Soda  Uii«  glass  which  does  not  have  a  precise  iodex.  Edmund  Scientific 
told  us  that  it  has  a  floating  index  that  varies  between  1.51  and  1.57.  They  can  guarantee 
a  value  between  these  two  values  but  they  cao*t  know  h  in  advance.  The  mode!  we  have 
chosen  is  a  dORmr  model  I)61»260. 

4.U  Ike  liidH  Emilliiig  Dloiie. 


The  LED  used  fks  the  specific^^^^Ksystem.  It  has  a  peak  emission  wavelength  at 
ggOmn,  the  maximum  value  for  alHmious  forward  cwient  is  SOOmA  and  with  such  a 
cttfient  foe  output  power  is  IQQinW.  Hus  LED  has  a  vtty  wide  angle  of  emission  and 
insofor  to  match  the  first  stage  specificatkm  for  an  angle  of  apeiture  of  0.11  radians,  we 
added  in  front  of  the  LED  a  little  igwture  that  would  only  let  pass  a  cone  whh  the  desired 
angle.  The  cafculation  for  the  diameter  of  this  small  apmure  is  shown  in  Hg.  22: 


lanas.^;  knowing asail«tanB:  ja»0.22dt 
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TraMtaUm  Stages 


t%we  21  Tic  awMriwg  systems 
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iX  11i>  liipareHtclivf  mifacc. 

Tbe  slipetiBg  U  coveuMi  wiHi  many  tbousands  of  minD  comer  cubes  <-  precisely  47jQQ0 
pel  iQuaiie  inch.  Such  sbeetiog  mterial  is  commonly  available  fifom  or  IMexie^^, 
Inc.  aMtUrottUaely  used  in  photoelectric  process  comtoL  Several  samples  were  aem  to 
us.  and  the  first  thing  I  did  was  to  test  then  behavior  by  sendhig  a  simple  laser  beam  on 
them(Fig.25):  ^ 

^  ff  tlettfireOective  wrfke 


Plpm  V.  Testiag  liMrelfaeaecUve  meet  with  a  LASER 

When  we  tested  the  sheet,  h  had  indeed  a  reiroreflective  behavior  if  the  retroieflective 
aur&ce  was  tilled  or  bent,  the  image  of  the  beam  was  st81  at  the  same  place  on  the  screen. 
Put  what  we  could  immediately  notice  is  that  the  image  spot  of  the  beam  on  the  screen  is 
sptead  and  that  leads  to  think  that  the  retroreflective  surface  indaces  scattering  in  one 
way  or  another.  And  that  might  have  critical  consequences  when  we  will  put  this  siirte 
in  our  system.  Moreo¥er  we  could  observe  a  ghost  image  coming  from  the  surfroe  and 
this  particular  im^e  could  he  assimilate  to  a  parasite  reflection  because  it  follows  die 
nMivements  of  the  sivfoce.  In  the  meanwhile,  we  measured  what  amount  of  energy  we 
could  collect  back  from  the  surfoce,  and  the  results  were; 

For  a  100%  amomn  of  incident  energy: 

Petroieflection;  45% 

Ghost  image: 

'ne  missing  amount  of  energy  is  apparently  lost  through  the  surfrce. 
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44.  Hw  CCD  caBwn. 

We  used  an  mfiraied  CCD  camera  provided  by  Artificial  Realky  Corp.  The  software  we 
used  to  acquire  the  knaget  on  the  computer  is  XCAP  Imagiitg  (fimited  featured  version) 
that  aOowed  us  to  capture  the  images  and  do  basic  analysis  on  them. 

S.  Meaniraiirals  8«d 

While  waiting  for  the  last  parts  of  the  purchase  to  hold  the  LED  in  the  rigtn  position,  we 
decided  to  use  a  LASER  as  a  light  source  wkh  the  setup  to  really  how  the  system 
was  behaving  and  to  make  some  observmton  that  could  be  useful  fbr  the  usage  of  the 
LED. 


5.1.  Tertihg  tiM  syfifiii  with  • 

Using  a  laser,  we  aligned  the  elements  according  to  CODE  V*s  specificatioiis  and  we  put 
the  CCD  camera  in  the  image  plane.  Then  we  ran  XCAP  Imaging  and  we  acquired  the 
image  of  the  LA^R.  The  image  we  get  is  shown  below  (Fig.  26a),  with  the 
corresponding  energy  distribiuioo  (Rg.  26b). 


Il8«clia.lp>«ae  of  the  LASER  hraufbUwsysieni  PlgmlA.ConeipoiMliog  energy  disiril^^ 


The  energy  distribution  above  is  obtained  thanks  to  the  software  by  aelectii^  arow  in  the 
image;  then  the  software  shows  the  energy  along  that  line  with  respect  to  the  pixel 


Appendix  D  -- 1 21 


involved-  TiMt's  wky.  lecause  of  the  speckle  iaduced  by  the  LASER,  the  curve  is  noisy. 
Adler  smooting,  it  is  dhown  in  Figme  27. 


I1e««  ]V.  Usage  of  a  PPT  filler  to  aooolS  Ibe  fiomer  OMvgy  diiiriki^ 

The  LASER  can  show  IIS  two  dtffcseiit  things; 

t  The  image  shape:  As  we  feated  it  when  we  analyze  the  behavior  of  the 
letfortflective  sheet,  the  hna^  was  spiead  along  one  dhection,  and 
imfoitunalely  this  was  the  dhection  of  the  hiture  ID  CCD  Hne.  Actually,  we 
should  get  a  veitical  line,  but  as  the  lenses  have  no  power  along  the  horfasonial 
dhectioii,  we  have  a  big  bhir  around  the  vertical  line. 

•  The  mapphig:  We  can  verify  if  the  mapping  of  the  huermediary  image  plane 
is  correctly  done.  In  Cki  the  system  nnist  be  invariant  in  one  dhection  and  in 
the  other  dhection,  the  image  should  follow  the  oliiect  Indeed,  the  system 
behaves  as  expected. 

54.  Viliii  llR  IJII  «i  a  IlfM 

The  first  thing  we  should  specify  is  that  we  use  the  LED  as  a  point  source,  but  we  must 
keep  in  mind  (hat  in  the  teal  system  the  aotnee  of  the  second  stage  is  not  a  LED,  it  is 
aheady  an  hNenaediary  image  of  that  LED  obtained  by  the  first  stage  of  the  tracking 
system.  So  in  order  to  match  the  specifications  of  that  huermediary  image/object,  we  put 
an  apertuee  in  hont  of  the  LED.  This  LED  was  a  very  wide  angle  LED,  so  we  had  to  put  a 
Mack  box  around  the  LED  to  prevent  h  fioro  directly  ilMminating  the  whole  system. 
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Then  we  positkHied  the  in  the  ctMiect  position  and  we  acquired  the  hnage  given.  We 
will  only  dhow  in  this  n^it  the  energy  distribution  of  the  image  because  the  Mack  and 
white  image  we  obtained  was  not  very  lefHesentative.  Immediately  we  notice  that  k  will 
be  difficult  to  exploit  the  results.  In  fact  we  need  to  detect  very  small  displacements 
(about  10pm),  and  the  curve  tqypears  too  flat  to  get  correct  demctkm. 


Ensny  DiilriNlInn  (iBD) 


flwl  Nwahtr 


nnwc  m  Eoetgy  in  the  tinige  pbne  utiag  Uw  USD 


We  computed  the  position  of  the  maximum  of  the  energy  on  the  image  plane  to 
determine  ntere  the  LED  is  on  the  image  line.  We  attenyued  two  methods:  the  detection 
of  the  cenlioid  the  energy  distribution,  and  the  detection  of  the  peak  erf  this  oiergy 
distribution. 


niese  measurements  and  tests  have  shonm  that  the  quality  of  the  retroreflective  sheet 
critically  affects  the  performances  of  our  systenx  The  mapping  of  the  image  plane  is  as 
we  expected,  but  the  scattering  that  is  produced  by  the  sheet  doesn't  ahow  us  to  pefform  a 
precise  deieotloo  of  the  LED  positioo  in  the  square  mtemiediary  image  pbiie.  We  are 
currently  inverigating  a  new  line  of  products  with  3M. 
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Design  of  an  anamorphic  fisheye  lens 
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Abstract:  The  design  of  a  90  x  30  degrees  FOV  anamorphic  fisheye  lens  is  presented. 
The  F/4,5  lens  maps  on  a  60  mm  square  detector  array. 
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Summary 


This  lens  was  designed  as  an  investigation  for  the  first  stage  of  a  larger  optical  system. 
The  first  stage  images  infrared  light  emitting  diodes  that  can  be  localized  within  a  90 
degree  (x-dimension)  x  30  degree  (y-dimension)  on  a  60  mm  square  array.  The  spectral 
bandwidth  of  the  sources  is  ISOnm,  with  20%  spectral  output  at  810nm  and  960nm.  The 
layout,  rayfans,  spot  diagrams,  and  the  modulation  transfer  function  are  presented  for 
performance  assessment.  Distortion  is  constrained  to  achieve  a  linear  mapping. 
Achromatization  of  the  lens  is  addressed.  Finally,  some  cost  issues  associated  with 
fabrication  are  discussed. 


Figure  1.  Layout  of  the  anamorphic  fisheye  lens  in  the  y  (top)  and  x  (bottom) 
dimensions. 

Methods:  Anamorphic  fisheye  lenses  are  most  commonly  designed  by  placing  an  afocal 
attachment  in  front  of  an  existing  symmetrical  fisheye  lens.  This  leads  to  sub-elegant 
solutions  due  to  the  bulkiness  of  such  a  front-end  attachment.  The  solution  we  propose 
utilizes  an  internal  afocal  sub-system  located  near  the  aperture  stop.  The  system  aims  to 
satisfy  30  micron  RMS  spot  size  over  the  FOV.  The  F/#  is  4.5  in  the  x-dimension  and 
13.5  in  the  y-dimension.  Distortion  was  constrained  using  a  chief  ray  in  the  merit 
function  to  achieve  a  linear  mapping.  We  have  chosen  the  lens  to  be  telecentric  in  image 
space  to  facilitate  the  mapping  requirements  of  the  following  optical  system.  The 
telecentricity  condition  was  constrained  using  two  chief  rays  in  the  FOV,  one  in  each 
dimension.  This  constraint  could  be  released  for  other  requirements. 

The  starting  point  of  the  design  was  chosen  to  be  a  symmetrical  fisheye  lens  designed  by 
Edwards  et  sd.,  (1993)  for  video  see-through  head-mounted  display  technology  (Rolland 
et  al.,  1994).  This  lens  consisted  of  two  meniscus  negative  lenses  typically  encountered 
in  fisheye  optics  and  two  doublet  lenses,  one  on  each  side  of  the  aperture  stop.  This  lens 
was  first  scaled  to  satisfy  the  focal  length  requirement  in  the  x-dimension.  An  afocal  lens 
made  of  four  y-cylindrical  lenses  was  positioned  after  the  aperture  stop  between  the  two 
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doublets.  We  constrained  in  the  early  design  stage  the  light  to  be  collimated,  entering  as 
well  as  exiting  the  afocal  assembly.  The  required  power  of  the  lenses  for  the  afocal 
assembly  were  computed  to  satisfy  a  reduction  ratio  of  three  in  FOV  between  the  x  and  y 
dimensions.  Most  importantly  for  cylindrical  optics,  the  afocal  constraint  should  be 
satisfied  over  the  entire  FOV.  Because  such  a  constraint  cannot  be  reached  with  afocal 
optics  (Wetherell,1992),  the  afocal  condition  was  released  to  balance  performance  over 
the  FOV.  The  focal  lengths  in  each  dimension  were  constrained  in  the  merit  function. 

Results:  The  layout  of  the  lens,  the  ray  fan  plots  and  spot  diagrams,  as  well  as  the 
modulation  transfer  function  are  shown  in  Figs.  1, 2,  and  3,  respectively 


Anarnorphic  Fish-eye  Lens  System 


Figure  2.  Rayfan  plots  (left)  and  spot  diagrams  (right)  of  the  anarnorphic  fisheye  lens. 

The  RMS  spot  size  is  less  than  30  micron  across  80%  of  the  FOV.  Looking  at  the 
rayfans,  it  can  be  observed  that  the  lens  is  essentially  limited  by  chromatic  aberrations. 
This  result  is  presented  after  optimization  with  varying  indexes  and  dispersion  glasses. 
Some  boundaries  were  set  to  limit  the  choice  of  glasses  to  second  grade  glasses  for  cost 
consideration  (Thompson,  1994).  Further  achromatization  of  the  lens  using  conventional 
optics  did  not  yield  much  improvement  of  the  rayfan  plots.  The  addition  of  a  binary 
optical  component  located  on  the  flat  surface  of  the  smallest  cylindrical  lens  in  the  afocal 
assembly  was  investigated.  Performance  was  significantly  improved  and  will  be  detailed 
in  the  final  paper.  Typical  cost  of  fabricating  a  cylindrical  lens  with  a  binary  optics 
component  is  5K-10K.  For  cost  effective  optics,  we  may  use  a  spectral  filter  located 
close  to  the  detector  array  to  further  reduce  the  effective  spectral  range  of  the  overall 
system,  which  is  a  viable  solution  in  our  case. 
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Figure  3.  Modulation  transfer  function  (MTF)  of  the  anamorphic  fisheye  lens. 
Except  at  the  upper  corner  point  of  the  FOV  that  we  included  in  the  optimization 
of  the  lens  to  insure  acceptable  quality  on  diagonal  points  as  well  as  on  the  two 
main  directions,  the  MTF  is  greater  than  20%  for  spatial  frequencies  less  than  28 
cycles/mm. 

If  fabrication  of  such  a  lens  was  to  be  pursed,  the  glass  selection  of  the  first  lens  should 
be  given  special  consideration  because  it  is  ISOmm  in  diameter  and  therefore,  the  cost  of 
fabrication  of  the  first  lens  can  significantly  increase  by  several  thousand  dollars 
depending  on  the  selected  glass.  Because  of  the  overall  length  of  the  system  (950  mm),  a 
folding  prism  with  64  millimeter  on  one  side  was  inserted  within  the  afocal  assembly  as 
shown  in  the  layout  in  Fig.  1 . 
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Appendix  F--h4i 


REPLY  TO 
ATTENTION  OF: 


DEPARTMENT  OF  THE  ARMY 

us  ARMY  MEDICAL  RESEARCH  AND  MATERIEL  COMMAND 
504  SCOTT  STREET 

FORT  DETRICK,  MARYLAND  21702-5012 


MCMR-RMI-S  (70-ly) 


2  Feb  01 


MEMORANDUM  FOR  Administrator,  Defense  Technical  Information 

Center,  ATTN:  DTIC-OCA,  8725  John  J.  Kingman 
Road,  Fort  Belvoir,  VA  22060-6218 

\ 

SUBJECT:  Request  Change  in  Distribution  Statements 

1.  The  U.S.  Army  Medical  Research  and  Materiel  Command  has 
reexamined  the  need  for  the  limitation  assigned  to  technical 
reports  written  for  the  following  grant  numbers: 

j 

96-C-6117  ADB233743,  ADB243030,  ADB259960 

94-J-4113  ADB251394,  ADB241076^,  ADB220530'^ 

Request  the  limited  distribution  statement  for  Accession  Document 
Numbers  listed  be  changed  to  "Approved  for  public  release; 
distribution  unlimited."  These  reports  should  be  released  to  the 
National  Technical  Information  Service. 

2.  Point  of  contact  for  this  request  is  Ms.  Judy  Pawlus  at 
DSN  343-7322  or  by  email  at  judy.pawlus@det.amedd.army.mil. 

FOR  THE  COMMANDER: 


Deputy  Chief  of  Staff  for 
Information  Management 


